After a brief hiatus, itâ€™s time to look at the Valio case again. This time, Iâ€™ll explain a few decisions behind our content management model. Iâ€™m sure some of this sounds familiar to almost every web site developer, although most wonâ€™t probably dive as deep as we did.
So youâ€™re developing an ASP.NET MVC web application. You whip up Visual Studio and crank out a few controllers, views and whatnot. Ka-zoom, you have the working skeleton version available in a day or so. At this point, what youâ€™re actually doing is that youâ€™re exposing your key business objects â€“ typically the database â€“ through a server and a browser.
A few days or weeks later you ship the app, and immediately a business user tells you that â€śActually, that segment of text is wrong. The correct one isâ€¦â€ť. You quickly update a view. Two weeks and thirty updates later, you grow tired. Not only do they want to change the text, but they also want to swap in new images. And one day, youâ€™ll get a mail asking you for a chance to slip in an additional information page. At a specified URI, of course. â€śCan I have it added to the navigation, too?â€ť
At this point, youâ€™re likely to have a few content delivery related customizations across your codebase. Your route table has a handful of entries specifically for campaign and info pages. Your version control history for the Views directory shows an increasingly high percentage of checkins related to one-off customization requests.
There will be a day when you find yourself asking: Should I just have started with a content management system (CMS), using extensibility hooks to implement your business functionality?
All web sites require some kind of CMS functionality: the ability to edit typical fragments of web content. Because of this, almost all web sites are based on a CMS. And there are so many of them; see the Wikipedia list.
At the other end of the spectrum, most web applications have very limited needs for content editing. Almost all have some requirements in this regard: even the most closed warehouse management tool often has a small area reserved for administrative announcements.
Most web projects fall between these two extremes. A typical e-commerce application is mostly an application, but it definitely needs content input: product descriptions, images, special offers etc. all need to be designed, tested and typed in on the go, without needing a developerâ€™s input. A complex and diverse collection of content (such as Valio) is by definition a site and needs to be editable, but it also has a huge load of functionality â€“ some of which may be considerably easier to implement using plain programming tools, not a CMS framework.
Picking our sides
When planning the technology strategy for the Valio project, we knew we were in trouble no matter what we picked.
There were requirements for considerable application-like functionalities including user-produced complex content (recipes), APIs for third parties, and demanding search rules. Simultaneously, we knew we would have to entertain a diverse group of content producers, from experienced web editors to amateur moderators.
|If we chose a CMSâ€¦ We would have at least a stub implementation for most of our CMS functionalities.
We would implement all our logic using the CMSâ€™s extensibility API. This might range from being moderately acceptable to extremely painful, but we probably wouldnâ€™t know before we tried.
We would have an authentication / user profile system available out of the box. However, most profile systems are not designed to be very well extendable. In particular, most donâ€™t support complex integration to external directories.
|If we wrote a custom applicationâ€¦ We would have a relatively straightforward task implementing all the custom things. Hard things are still hard, but weâ€™d be able to estimate them with reasonable confidence.
We would have to write all the CMS tools by hand. Given our reasonably complex set of requirements (page previews, limited versioning, web part â€“like composability, customizable URIs, page templates), we knew this was going to be quite a few lines of code.
We could do whatever we want with the authentication. Of course, that meant doing it all by hand, from square one.
As you probably know by now, we picked the custom application route. The CMS side had its perks, but we decided against it for the following reasons:
- The client had a fairly particular vision of the content management model. While they didnâ€™t have stated requirements for the administration UI, we knew there were quite a few practical requirements, particularly regarding bulk moderations, that were not typically sufficiently implemented in CMS packages.
- While we had lots of CMS experience, we also had the requirement of using Microsoft technology, and an inner desire to use MVC to enable very fine-grained HTML output management. We also preferred an open source approach to ensure we could deliver everything we wanted. That left us in a situation where none of us had experience on a CMS that would match the requirements. Since evaluating CMS extensibility is very time-consuming, we didnâ€™t want the additional risk of eating our precious days.
- On the other hand, having lots of CMS experience (including developing two of them) gave us a head start on designing the necessary infrastructure. Thus, we felt less intimidated by the challenge of creating our own tooling.
At the crossroads?
If youâ€™re facing the same choice, I wouldnâ€™t blindly recommend following us. We have been met with plenty of criticism and surprised faces when telling this story. Many people consider custom app writing a symptom of the NIH syndrome, particularly on a field as well established as CMSes. Also, it is a non-trivial exercise even if you have a good idea on what youâ€™re doing.
The key lesson here is to play to your strengths, and choose by the project type. If you have a team with experience in a particular CMS and you have complex content management requirements, that particular CMS is likely to be a good idea. Then again, if all your users need is a single bulletin board for announcements, taking on a CMS framework is probably a hugely unnecessary piece of extra baggage.
However, one important thing is schedule predictability. If custom code and a CMS seem equally strong, consider any pre-baked system a risk in terms of change management: you quite likely cannot predict all its design constraints in beforehand.
For example, the requirements for Like button throttling in the Valio case were discovered reasonably late in the project, as was the moderation workflow for user's recipes. Most CMSes donâ€™t offer smooth customizability in scenarios like this, and thus a small requirement can suddenly result in a larger refactoring, perhaps writing a module that replaces a part of the CMS itself. You would also be excessively optimistic in thinking that relatively obscure elements such as community content workflows would – or even could – be defined before the project.
The diagram below illustrates some of the design aspects and their weight in a totally customized scenario as well as a CMS-based one.
Not all CMS platforms are equal. The right end of the axis represents the versatile but hard-to-extend CMS solutions like SharePoint. The middle section of the chart represents CMS stacks that are more like toolkits for do-it-yourself site development; there are plenty of open source CMSes that are unfinished enough to be called such.
We picked what we picked because of 1) the requirements we had and 2) the people we were. Neither is irrelevant, and this is a key takeaway: A different team might pick an entirely different solution for the same requirements, and they might quite well be right.
You wonâ€™t have an easy time deciding on this: itâ€™s a complex architectural choice, and estimating the actual impact of either option is reasonably hard. In the next post, Iâ€™ll discuss the key technical choices of our CMS implementation (on class/table level) to give you an idea on what sort of challenges you might be facing and help you in the process of gauging your cliff.
Pretty close to half a year ago, Microsoft held the largest annual developer + IT Pro event in Finland, TechDays 2011. In six more months, itâ€™s happening again.
As I was considering the various topics I might talk about, it always took me back to thinking about the years Iâ€™ve been talking there. What did people like? What kind of topic would interest people? How can I be better? My talks generally seem to fill a room of 100-200 people, but what do I have to say thatâ€™s worth 100-200 hours of Finnish developersâ€™ time?
The TechDays speakers have had almost nonexistent visibility at the feedback gathered from the event. Well, that is, until now. Allow me to present the TechDays 2011 Feedback Infographic: (click on the image for additional resolution)
Challenges for TechDays 2012 speakers
There are plenty of conclusions one can draw from the data. The one that struck me the most is that people want highly practical information, but not without the theory. Case presentations beat pure theory hands-down, but the most popular sessions were the ones with high-energy presenters, a sound theoretical basis and a continuous pummeling of practical demos.
As Iâ€™m left pondering this, I want to offer three TD2012 challenges for fellow Finnish speakers:
- Sessions in Finnish scored 0.27 (on a scale of 1..5, thatâ€™s a lot) lower than sessions in English. Many attendees seem to gravitate towards the heavily rehearsed tracks coming from foreign travelling speakers. This must stop. Even though I think many of the travellers are absolutely great, Finnish technology professionals must be able to be more relevant and interesting*.
- Developer sessions scored a 3.61, which isnâ€™t particularly bad, but itâ€™s not good either. Many developer sessions are far too much based on the equivalent PDC/TechEd/Build sessions, perhaps even slides. Let me repeat: people want practical information and your experience with the theory. Letâ€™s do better this year.
- Read Scott Berkunâ€™s Confessions of a Public Speaker and either Presentation Zen or Slideology. Reduce the amount of verbiage in your decks and talks, and replace it with raw energy.
*) While berating the state of Finnish speaking, I must tip my hat off to Sami Laiho, a local Windows MVP. As stated in the infographic, the man did five different presentations, scored a 4.43 average (which would, had Samiâ€™s presentations been a single track, been the most popular track of all TechDays), and pulled off the most popular presentation with a stunning 4.58 score. Oh, and the 4.58 was done on day 2 first slot, which speakers typically shun as the previous nightâ€™s attendee party is considered to hamper the contact with the audience. Blah blah.
A few words on the data used:
This post is based on a data dump of TechDays 2011 feedback I have received from Microsoft. No personally identifiable information was ever transmitted to me. The infographic has been cleared for publication by Microsoft, but such publication probably indicates no endorsement or broader approval. I do not have permission to redistribute the raw data, so questions for further info may or may not get addressed.
This blog post contains conclusions and opinions, which are naturally mine. On the other hand, the infographic is based on objective, raw event data with the following exceptions:
- Some questions have been grouped together for better illustration (namely, the food and venue/organization ones).
- I have manually divided the presentations into â€śtypeâ€ť categories (theory, practice, case, lecture series). This grouping is non-official.
- Grouping presenters was done by me. Deeming people â€śprofessional speakersâ€ť or â€śnobodiesâ€ť involved subjective analysis.
- The textual description and the visualization is mine. The original data sheet nominates no kings, nor were pizza slices actually available.
Thanks for listening â€“ and letâ€™s hear your thoughts on TechDays 2011 and its feedback :-)
One year and a couple of days ago we kicked off Offbeat Solutions, our consulting company of four people. We are still the same four, but far bigger four: we know more, weâ€™ve done more, weâ€™ve seen more. Here are a few insights from the first year.
3 things we nailed
Customer satisfaction. We took deliberate risks in picking our projects, but didnâ€™t overdo it. We managed to deliver challenging things with a positive attitude, and people around us seem quite happy. We wanted to become trusted advisors to our clients, and for most, we did.
Not hiring anyone. It would have been easy (and even profitable) to get new people along; we had exactly the right cases for that, and we could have done so with acceptable costs. But had we done that, we wouldnâ€™t have this strong unity, the ability to take on future incertainty with extreme trust in ourselves. There will be time for new hires, but it is not yet.
Setting up the office late enough. We discussed this office thing early on, but executed right at the 12-month mark. And damn, am I happy about it. Having worked together for a year, we had a reasonably good idea on what weâ€™ll need to be productive (and have fun). Had we gone for the place instantly, we would not have had the resources to make it good. We probably still wouldnâ€™t have moved out, simply because of the effort it takes. We couldâ€™ve been cheaper on this, but probably not much more efficient. More on the office thing in a later post.
3 things we sucked at
Teamwork rigor. This may come as a surprise to you, particularly if you worked with us. Still, internally, we feel that we havenâ€™t been at the top of our game in working together. Sure, we beat the average organization hands down, but we find ourselves insufficiently seeking support from our each other â€“ even when we obviously have a great chance. One of our 2011-2012 goals is to kill all procrastination by not only allowing, but demanding requests of assistance when things seem to slow down.
Technology breadth. We wanted to focus in working with the Microsoft stack, but take in the best parts from other technology families. Sure, we leverage lots of open source and external tools, but weâ€™re nowhere near good enough. We write great ASP.NET MVC apps, but after our second year, we should have zero problem writing the same with Node.js, Rails, Akka or whatnot. And even inside the Microsoft stack, thereâ€™s plenty to learn.
Sharing with the community. We had the vision of being a storyteller, a company whose experiences would be publicly shared. We did some of that (first and foremost, our posts on valio.fi), but we were nowhere near as loud as we wanted to be. Given that we consider the year a great learning experience, itâ€™s simply a shame that we havenâ€™t found the time to share more.
3 things that made the year worth living
Work/life balance. Canâ€™t say it was perfect, and we certainly worked too much at one time. Still, we have exploited our freedom of choosing where and when we work. There were frustrations, but few of them were really caused by too many hours.
Sufficient diversity. We did quite a few things. We worked on CRM strategies and security reviews. We wrote code and helped manage projects. We tackled SharePoint, Azure and PHP all alike. We worked with startups, SMBs, large enterprises, NGOs and the government. All great learning experiences.
The fun of it. We developed a culture – of course, not really intentionally. We developed a (bad) taste of office music, we printed out big pictures of legendary artists on the walls â€“ and naturally, we started wearing military uniforms made of rubber during code reviews. We were supposed to be offbeat, right?
Itâ€™s been a blast, but weâ€™ve grown rather critical about ourselves. Our start was way smoother than we expected, and that gave us a somewhat false feeling of security.
Weâ€™ve been fighting it for months now. It would be easy to bury ourselves into a few big customers, learn to understand their business deeply and become irreplaceable. But then, a few years down the line, weâ€™d find ourselves fat, lazy and unlearned. Pretty much the opposite we planned originally.
Plan for 2011-2012: Work on the most difficult and ugly things we can find. Win ourselves every day. Weâ€™ll be back to tell you how weâ€™re doing.
This post is an exception from the English majority of my writings. I will be summarizing the Finnish .NET User Groupâ€™s event on F# and functional programming, and will do it in Finnish. In summary, a great event!
NeljĂ¤s SANKO-tapahtuma jĂ¤rjestettiin 9.6. ohjelmistoyhtiĂ¶ Reaktorin tiloissa Helsingin keskustassa. Tuomas Hietanen ja Rami Karjalainen alustivat kertomalla kokemuksiaan funktionaalisesta ohjelmoinnista ja esittelemĂ¤llĂ¤ F#-kieltĂ¤. Kolmen tunnin tiukan esityksen jĂ¤lkeen ilta jatkui ruokailulla ja saunomisella. Paikalla oli kokonaisuutena kolmisenkymmentĂ¤ ihmistĂ¤.
Tuomaksen ja Ramin esitykset voisi tiivistĂ¤Ă¤ seuraavasti: He nĂ¤kivĂ¤t funktionaalisuuden olennaisena positiivisena trendinĂ¤, ja kokivat F#:n erittĂ¤in vakavasti otettavana kielenĂ¤ vĂ¤hintĂ¤Ă¤nkin liiketoimintalogiikan ja datan toteutukseen. He olivat pitkĂ¤Ă¤n tehneet yhdessĂ¤ vakuutusalan hanketta, jossa F# oli keskeisessĂ¤ roolissa.
YleisĂ¶ssĂ¤ oli vahvaa kiinnostusta aiheeseen, ja Tuomas saikin selittĂ¤Ă¤ funktioteoriaa ja muita periaatteita varsin pitkĂ¤llisesti. KĂ¤yttĂ¶kokemus F#:sta itsestĂ¤Ă¤n oli harvassa, mutta C#:lla funktionaalista ohjelmointia â€“ siis LINQ:n tehokĂ¤yttĂ¶Ă¤ â€“ oli kyllĂ¤ harrastanut monikin. Funktionaalisuutta harjoitellessa F# nousi C#:a suosituimmaksi mm. vahvemman tyyppi-inferenssin (tyyppien automaattinen pĂ¤Ă¤ttely, jolloin koodissa tarvitaan vĂ¤hemmĂ¤n turhaa tyyppien toistelua ja generics-kulmasulkeita) ja interaktiivisuuden takia. KĂ¤ytiinpĂ¤ esitysten lomassa myĂ¶s keskustelu, jossa pohdittiin testivetoisen kehittĂ¤misen (TDD) arvoa ja sitĂ¤, onko itseasiassa interaktiiviseen REPL-tyĂ¶kaluun pohjautuva interaktiivinen kehitysmalli tehokkaampi.
Kiitos kaikille osallistujille ja esiintyjille â€“ ja tietysti myĂ¶s Reaktorille sponsoroinnista. Ja hei, Reaktorin lĂ¤snĂ¤olo .NET-kentĂ¤llĂ¤ tarkoittaa siis sitĂ¤, ettĂ¤ Suomen parhaana tyĂ¶paikkana monesti palkittu softatalo on nyt auki myĂ¶s Microsoft-osaajille. Se on merkittĂ¤vĂ¤ asia, enkĂ¤ sano tĂ¤tĂ¤ vain siksi, ettĂ¤ sain hyvĂ¤Ă¤ ruokaa. :-)
In my series on valio.fi deep dives, Iâ€™ll return to the original fields of Offbeat and discuss some backend solutions. We wonâ€™t dive into the code today, but will take a look at some hard architectural choices.
Essentials of your ORM decision
We chose to build a reasonably large web site using NHibernate. A few members of the Microsoft developer crowd immediately asked: â€śWhy not Entity Framework?â€ť Meanwhile, a few of our developer friends had a lengthy discussion deeming ORMs worthless and unnecessarily complex.
There are really two subquestions here:
- Should we use OR mapping to make data access more straightforward?
- If yes, which OR mapper should we use?
Let me be absolutely clear on this one: I think both those questions are important to consider honestly. Many object-oriented developers feel that automated object/relation mapping is an essential part of developer productivity. I say thatâ€™s a valid point, but somewhat self-supported: ORM is important if you arenâ€™t good enough at performing data access without it.
In this post, Iâ€™ll go over our decision making process, and then look back at where our use of NHibernate succeeded and failed. Mind you, the preceding emphasis is not a style issue: We take full responsibility for both our successes and failures, and do not intend to blame or praise NHibernate unnecessarily. It was just lying around, we used it.
To ORM or not to ORM?
This question was at the top of the list a few years ago. Now ORMs are mature enough to be the default choice for most developers, thus eliminating the question. Although the option of working without an ORM was debated, we sort of wanted one anyway. Our decision was based on a few facts:
- We had a non-trivial (although not particularly complex by any measure) business model, consisting of a few dozen entities.
- We had reasonably straightforward query requirements, mostly based on foreign key navigation.
- We had a mostly-read model, which ORM could naturally support by providing features such as identity mapping and caching.
- We had relatively few complex write operations and thus thought weâ€™d get away without encountering the worst parts of mappings.
All these are good signs for ORM use. We werenâ€™t entirely right in our predictions when making these decisions, but looking at it afterwards, Iâ€™m not sure we could have done better.
Picking the titans to fight
The next step was to compare Entity Framework to NHibernate.
Wait, why exactly these two? Welll, there are two factors: First, we had NHibernate experience available. Second, EF was the Microsoft Option. Well, we also had experience in other ORMs beside NHibernate and EF, but after considering our maturity and expertise in each, decided to limit ourselves to these two.
â€śThe Microsoft Optionâ€ť? Itâ€™s usually the official platform-integrated benchmark solution. Razor for view engines, EF for ORM, WCF for REST implementations and so on. Often not the best option, but the best known one. Should always be one option when picking a technology â€“ being well-known is a big bonus, plus your software design never passes even a cursory review without you being questioned about ignoring the default option.
Donâ€™t confuse that with picking the Microsoft Option blindly. We as a community of Microsoft developers are plagued by a disease that encourages choosing the MSFT solution every time, regardless of factual benefits. That is just as bad as ignoring the Microsoft option without thought.
Being included in the platform is a merit that justifies being shortlisted into any technology comparison, but definitely not winning them.
The fact that we chose only those two products for comparison seems arbitrary. This is an important point. It is entirely possible that there could have been a better solution we just didnâ€™t know of. We decided to limit ourselves to these two because we considered the cost of exploring other options greater than the potential benefit. For any given technology, the benefit of evaluating a new option is about this:
B = p * ( AE â€“ LC ) â€“ EC
In the equation, the benefit B comes from subtracting the learning cost LC from the added efficiency AE and multiplying the result with p, the probability of this technology actually being picked. Furthermore, you need to subtract the evaluation cost EC from the benefit (although conceivably EC and LC would somehow merge, but weâ€™ll keep it simple now).
Now, consider this: For an OR mapper, sufficient evaluation of an unknown variant takes at least a day. The trivial stuff you do in an hour, but going through all the complex scenarios (custom field types, xml handling, transactions, caching, cascading deletes, â€¦) takes at least a day. The learning cost of doing all that properly is at least 5-10 days, particularly in a team of multiple developers. Even if we considered that some Acme ORM would be a contestant equally favorable to EF and NHibernate (i.e. p = 1/3), we would get:
B = 1 / 3 * (AE â€“ 5) â€“ 1
Quickly looking at it, we realize that with AE < 8, we would probably gain no benefit at all. Even if we estimated (before evaluating anything!) that a new technology saved us a whole month (20 man-days) of effort, our estimated benefit would be 1/3 * (20-5) â€“1 = 4 days. Would four days actually offset spending the one day in evaluation? And if we chose that, weâ€™d gain 15 days but take a whole bunch of risk in exchange. How well can future generations of maintenance programmers benefit from this choice? And mind you, 1 day for eval, 5 for learning and 1/3 for usage probability are very optimistic estimates.
Iâ€™m very much pro-evaluation when choosing technologies, but you have to know when to stop. Considering that we didnâ€™t plan to do many man-months of data management effort anyway, the potential gains never felt important enough to warrant the exploration effort. Thus, we focused on things that we found more productive.
The discussion above has nothing to do with ORMs per se, but ORMs are a good demonstration of this principle: since each stack does roughly the same things, knowing one of them relatively well is an extremely strong argument for picking it up and getting to real work.
Entity Framework vs. NHibernate, real arguments
We chose NHibernate for a few reasons. The key arguments were:
- NHibernate is more mature (and at the time the decisions were made in September 2010, the difference was even greater)
- NHibernate was better equipped for distributed caching (EFCachingProvider labels itself as a â€śsampleâ€ť) and manipulation of fetching strategies.
- NHibernate + FluentNHibernate provided a reasonably approachable way to configure various complexities; EFâ€™s approach of hiding some of this in a design diagram isnâ€™t entirely without problems.
We did give EF serious consideration â€“ finding people with NHibernate competence isnâ€™t trivial, and as such, NHibernate is a hindrance to the further development. However, given the constraints we had, we had more confidence in choosing technology with a longer history (particularly on the Java side).
Our data layer architect Lauri* spent considerable time tweaking and tuning the data access mechanics â€“ but most of this was done to create a smooth way to organize DAL code and make it accessible through the IoC mechanism we had. The bulk of that work was unrelated to NHibernate in itself, though much of it would have been different if we had opted for raw data access instead.
Most significant architectural headaches we had with NHibernate involved some of the more complicated structures. We have an everything-in-one-table â€“modeled class hierarchy with a discriminator field and an XML column for subclass data storage. Getting this to work wasnâ€™t exactly smooth, nor was parsing the same column into two distinct fields. I may get back to this in more detail if thereâ€™s interest.
Performance problems are one of the most common fears with ORMs. For us, query performance wasnâ€™t really a problem â€“ reckless use of lazy loading did result in some issues, but this is more a result of underconstrained data usage than a problem related to choice of ORM itself. We have a few scenarios where NHibernateâ€™s query cache manipulations actually slow down some operations, and we had to work around that. Most of the time, NHibernateâ€™s fairly intuitive system of specifying fetching strategies worked like a charm.
Perhaps the most head scratching was caused by cascading delete/updates and ordered lists with index values in the database. Each of those required special trickery and effort, and particularly complex update scenarios were considerably difficult to verify as working correctly. These are the definite drawbacks in using an ORM â€“ the abstraction really gets in your way far more than it benefits. Of course, such extremes do not typically occur in the vast majority of the program code.
*) We never appointed a data layer architect. He just grew up to that role. We like it that way.
NHibernate did a good job, but it did exhibit all the problems of an ORM. Man, that stuff is hard! I mean, both EF and NHibernate make it very easy to whip up the simple demo application. If you have a reasonably straightforward business application with few concurrency issues and low performance requirements, chances are an ORM will totally save you. If you have a high-volume application with severe execution time constraints, you will probably encounter details that you never wanted to hear about.
I think the cost of using direct data access (data readers, commands and the like) is relatively linear compared to the project complexity. ORMs are much easier for a long time, but once you get to the hard end of the scale, their intricacies really start hurting you. I think this is particularly true for Entity Framework; for NHibernate, the availability of the source code provides some relief â€“ or just just an extra measure of confidence – for the most extreme hacking scenarios.
Donâ€™t pick an ORM to make things easier unless youâ€™re dead certain that youâ€™ll stay in the comfort zone of your competence. Of course, if youâ€™re the resident EF/NHibernate surgeon, youâ€™ll enjoy the benefits far further than a team full of newbies.
Personally, I wouldnâ€™t change our basic decision to use NHibernate. However, I would be more judicious about our options in each particular point of the application â€“ using an ORM somewhere does not necessarily mean it is the best thing to do everywhere. Also, I would make sure the ORM understanding is better disseminated inside the team â€“ although not everybody needs to be able to fine-tune those fetch optimizations and lazy loads, everybody should understand the issues clearly enough to spot possible problems in reviews and other pre-deploy phases.
Up next, a week of vacation. Later in June, weâ€™ll look into the back-end in more detail.
A step aside from the deep dives!
The valio.fi project is now also presented in the famous Finnish Vierityspalkki.fi blog. If you can read Finnish, check out my guest post detailing the site and project from a general web developer (non-backend) perspective.
Meanwhile, deep dive #4 is cooking. The topic will be OR mapping, and Iâ€™ll post it on the next week. My apologies for the delay inbetween â€“ you wouldnâ€™t want the details (dental surgery and all).
After my previous post on review policy, letâ€™s have a look at the tools we used for reviewing code.
I have found there to be two approaches to reviewing code. Letâ€™s do a quick comparison first and discuss the tools of the trade next.
|Patch-based review||Social review|
|The key concept here is that a developer prepares a set of changes, and publishes it for review. The reviewer focuses on the changes (i.e. â€śthe patchâ€ť).
Reviews typically happen through a system (from an work-item attached unidiff file to a full web-based system).
Comments on the review are often formalized documents, email messages or if a specialized system is used, its comment entries.
|The developer presents the code to the reviewer. Some do this in a meeting room where multiple people review simultaneously, most of the time it happens pairwise at the devâ€™s workstation.
Reviews tend to focus on code understanding as a whole, and discussion about the code happens in-review. Written documents are sometimes produced, but most organizations rely on verbal communication and the developerâ€™s personal notes.
Mind you, this is not review theory, but a very important split when looking at the tools. Most projects can easily benefit for the social approach because it requires less tooling and technical sophistication. My experience is that neither catches everything, and the ideal approach might involve both â€“ but reviewing everything twice is often too expensive in terms of time.
We did both, and the social one was easy
Social reviews happen naturally when a team communicates well. The rigidity of the team organization usually determines if social reviews are specifically scheduled, if comments and their responses etc. are tracked, what kind of audit trail is required to accept a social review from a project management perspective and so on.
We used a reasonably liberal system. We required reviews (see the previous post), but accepted socials just like more formal, patch-based reviews. Either form of review was sufficient as long as the reviewer and developer felt like it. And we had no tools for social reviews. We had no document templates, no scheduling, no way to track the comments.
It worked for a reasonably well-bonded team of nine, and it might work for you. Donâ€™t use lack of tooling as an excuse to not review socially; if tools are necessary for you, youâ€™ll see the need to find them as you go.
Patch-based is a different beast
When you adopt patch-based reviewing, pay attention to the tools. Iâ€™ve done this for years with the pure unix diff/patch approach, and would not recommend it for development today. There are tools available, and looking at them is a good ideaâ„˘. We used Crucible, but let me be crystal clear: We are tool pragmatists. We used Crucible because it was easily available, relatively cheap and worked for us. We donâ€™t claim it was the best solution even for us, much less you. If you have the liberty of choosing your tools, do check it out â€“ but also look at the competition.
But before you go on a spree of tool comparison, let me remind you that your review abilities, processes and social capacity far overwhelm your selection of tools in terms of determining your success with code reviews. If youâ€™re setting up a review system from the scratch, you might spend a day on finding the appropriate tooling, but spend four more thinking about what, how and why are you reviewing.
Enough with the intro, go Crucible
The rest of the post will now discuss our patch-based reviews and the use of Crucible (to which the repository browser FishEye is tightly integrated to). First, Iâ€™ll walk you through the typical elements of a review, then discuss some of the tooling experiences.
As discussed before, we were in the post-checkin camp. Therefore, our reviews started with the developer committing the code to our version control (Subversion). We had Crucible and FishEye set up to poll the repository on regular intervals. Thus, on our Crucibile web site, we continously saw the stream of checkins listed.
For any of the checkins, one could create a review and pick the reviewers desired. There are multiple ways to configure and use Crucible; we used an approach that forced the author to pick the reviewers, but the reviewers could ask for more people to join. Although we encouraged an open review culture where anyone could review anything, we felt it necessary to identify the reviewers responsible â€“ this would give everybody a clear idea on whatâ€™s on their plate (â€śOh, I have these four things to review todayâ€ť).
In Crucible, each review request contains the files to be reviewed. Since Crucible is integrated to the version control, these files are picked right from the source control tree, and can be either diffs (changes between two versions) or whole files. In any case, the files are shown in the browser, with coloring for the diff elements.
The whole point of reviewing is to get feedback on the code. Crucibleâ€™s feedback system is based on two concepts: comments focused on a certain code block (one or multiple lines) and comments on the whole review. The former is an approach to very specific feedback. Crucible also allows replying to the comments, thus generating nested comment/discussion trees.
Side note: As you can see from the screenshots, we had English as the code language but Finnish as the working language, and reviews were conducted in Finnish. We didnâ€™t consider find this to be a problem. For us, a code comment is documentation for the future developer (potentially not knowing Finnish); a review comment is discussion between the team now (which is totally Finnish).
We make no recommendation on this – apart from keeping the commenting/reviewing bar as low as possible. Finns generally comment more eagerly in Finnish, but your mileage may vary.
The review process is fairly simple: Every requested reviewer goes through all the files, adds his comments and marks himself as complete. Once every reviewer has completed the review, the author goes through the comments, replies on them, fixes the code and finally closes the review. Rounds of discussion can ensue inbetween, but are often better handled outside Crucible (see below).
Experiences and best practices
Keep your reviews small. In fact, keep your checkins small. If your checkins are small, your reviews are too. Fix one aspect of code in a single checkin, and then review that. Even though Crucible encourages the model of picking a source control checkin and clicking â€śCreate Reviewâ€ť, you can and should split checkins for review. Iâ€™ve written about this in detail back in 2004, so why repeat myself?
Review layers of trust or reuse separately. For example, if you develop a feature that needs new general use tools you implemented in the tools library, consider reviewing the tools separately. Separate review of the tools forces more focus on the general usability of the tools as the time-pressed reviewer doesnâ€™t have the inclination to just look at how they function at the specific call site. However, you donâ€™t necessarily want to take this too far; reviewing user interface and related data access in one go is probably a good idea.
Review unit tests. If your code has unit tests, include them in the review â€“ and review them. In fact, quite a lot of attention should be paid to the tests, as they are a good indicator of various code issues. Reviewer should understand the tests thoroughly and then analyze their thoroughness â€“ are all error paths reasonably tested? This approach provides an excellent way to spot bugs that you would very easily skip when just reading the actual implementation code.
Donâ€™t make reviews your discussion board. Prefer short comments, debate face-to-face and then summarize the result in a short comment. Chatting through Crucible is inefficient, and while reviews can have considerable short-term documentary value, this value is not improved by including all discussion. Including the relevant results of a discussion provides the best balance.
Combine review methodologies to reduce pain. An author about to check in a batch of changes should consider ways to improve the review experience. Since diff-based reviewing is based on line-by-line change tracking, certain types of changes make diffs very hard to read. For example, renaming a member in project-wide use causes scattered diffs for hundreds, maybe thousands of code lines. Such a diff is very cumbersome to review properly (i.e. verify that nothing else has changed); itâ€™s usually much better to do the rename together to eliminate the need for after-the-fact reviewing.
Donâ€™t be afraid to take another spin. Every now and then, a review results in considerable code rewriting. This doesnâ€™t mean the original code sucked; sometimes itâ€™s just that the review reveals things that were not accounted for (a similar construct elsewhere in the project, a new requirement or whatever). If the code must significantly change because of the review, reviewers should be comfortable asking for another round (â€śmake these changes and hit me with another review requestâ€ť). Some may consider this unnecessary supervision and lack of trust for the code author, but they are the ones who misunderstand the key reasons behind reviews. Review until you have confidence in the code being fixed properly.
Up next: A dive into the backend technology, for a change.
What could be important enough to interrupt the flow of posts on practical project experience? It is the reorganization that will throw every .NET developerâ€™s favorite ĂĽbergeek to a new position.
Yes, Scott Guthrie is leaving his post as the head honcho for many things such as IIS, ASP.NET, Silverlight and a slew of other products. His new task? Make sure the developers love Azure and get a solid development story on Microsoftâ€™s cloud platform.
What does this mean?
- ScottGuâ€™s blog will probably focus more on things related to cloud development (including private cloud). The division heâ€™ll be working in delivers database services, BizTalk, AppFabric, WCF and workflows.
- There will have to be new champions for his previous products. Perhaps not directly in the terms of organizational position, but in the terms of being the public face for said products. Scott Hanselman and others will probably come to the rescue web-wise.
- Somebody else will probably keynote MIX12. Then again, PDC11 will probably see even a more empowered ScottGu preaching the Azure religion.
One thing Iâ€™m a bit worried about is the openness development that has been set well into pace in Guthrieâ€™s previous organization. Co-operation with the jQuery team, integration of new open source technologies into the Visual Studio / .NET mix, growing support for releasing previously closed source, very open and straightforward relation to the developer sceneâ€¦ All this has been highly valuable.
Iâ€™m sure the Gu will take much of the goodness to his new org, but Iâ€™m less certain if the goodness will remain in the old one. At the same time, all this makes sense: ASP.NET and Silverlight are very, very mature compared to the development story on Azure. Itâ€™ll be interesting to see his impact over the coming months. Also, given the somewhat inherently closed nature of the cloud, the next steps of the openness story are worth looking forward to.
In this second episode of my postings on valio.fi, Iâ€™ll look deeper into the concept of code reviews in the project.
Pre-post note: No, Iâ€™m not following a logical order with these. Rather, Iâ€™ll try to answer the asked questions as soon as I can while still maintaining a readable approach to the project details. Front-end posts will be interleaved with back-end magic, and there will be a dose of project management sauce inbetween. If you want to hear about something, ask for it in the comments.
In a project with numerous developers â€“ in our case a scrum team of nine â€“ the question of spreading code knowledge and ensuring certain quality bar tends to arise. There are numerous ways to tackle this, but we picked two principal approaches: Shared physical working space and code reviews.
Peer reviews vs. architect reviews
There are basically two alternative approaches to picking the reviewer. Either you have a reviewer/architect role (which may be rotating, but often tends to stick), or you can have a system of free peer reviews, i.e. anyone can review anything as long as they feel sufficiently competent about it.
Wanting to emphasize the equality of team members, we naturally chose the peer review model. However, few teams have a totally balanced review graph. Usually some people end up reviewing more code than others. This is partially a function of seniority, but also dependent on character, preferences and random factors (such as knowledge of a particular tricky subject).
Ours wasnâ€™t balanced either. A couple of persons ended up being a part of most of the reviews. Still, we found the system to be fairly optimal: it did distribute a lot of knowledge, everyone had their code reviewed by several others and everyone reviewed somebodyâ€™s code.
As with any project with more than a few people and a duration of months or more, writing down the intended methodology felt wise. Thus, we did have written review guidelines, although they were just that – common sense was allowed to rule over these idealistic methodology statements. Here is a summary of our guidelines with some examples and clarifications in parentheses:
Some notes on the guidelines
These rules, as written above, were applied at the start of the project. During the final few sprints, we relaxed the guidelines and focused on shipping.
No, wait, maybe that was what you expected to hear. Actually, we tightened the rules a bit for the final three sprints (six weeks) or so. Mainly, we merged the two first headings; essentially, we started to require reviews on all non-trivial changes to the code base. We also pushed this to our project management system; a sprint backlog item was not marked complete until it had passed review (for whatever definition of â€śpassedâ€ť; we still allowed the author to decide whether review comments warranted code changes or not).
As you can see, we relied a lot on code authors. We allowed everybody to request review from anyone, meaning that we didnâ€™t â€“ nor did the project management â€“ get involved in how much time was spent in reviews. And quite some time was spent, although we didnâ€™t log any specific numbers. Personally, I spent almost half of my time reviewing code towards the end of the project, although admittedly I was one of the most active reviewers.
Forced vs. recommended: Before or after checkin?
There are basically two stages when you can do a review. Many open source projects take a fairly strict path: Only a select few people can commit or authorize commits to the source control, thus essentially creating a tight net for reviews. Often, in the largest projects, nobody commits anything without somebody else looking at it first. This makes sense when nobody can know the code authors. The other alternative is to allow free commits, but review before releases or other milestones.
We were totally in the post-checkin camp. Everyone could commit anything, and we had no system in place to track if people actually did request reviews (until late in the game when we made it part of the completion criteria).
If you have the right culture, reviews are free
So yeah, we had a very author-controlled approach to reviews. But we also had a very open code quality culture, and it was not uncommon to question the validity of somebodyâ€™s approach over lunch. The debates could get quite heated, and we often spent a lot of time on issues that didnâ€™t, in the end, have enough weight to warrant such use of time.
However, it all served a purpose. Our somewhat excessive quality discussions created a culture where writing bad code â€“ or skipping reviews â€“ wasnâ€™t particularly lucrative. You never got publicly lambasted for a stupid bug in unreviewed code, but everybody felt better collectively owning code that had passed a review: a bug in well-reviewed code was a team mistake.
The reviews definitely had a time cost. Their positive impact is very hard to measure, given that we couldnâ€™t know what the amount of bugs would have been without the review net. We caught some in the reviews, but probably prevented ten times more by conveying knowledge and setting examples during the review process.
A couple of weeks after shipping the site, many people kept wondering about the low amount of bugs discovered right after the launch. This sentiment, no matter how unmeasurable, is one of the key indicators of our success in terms of quality.
A perfect success?
Far from it. First of all, our front-end code wasnâ€™t nearly as meticulously reviewed, for a variety of reasons (although the same attention to quality did apply, it wasnâ€™t as collective). There were definitely scenarios where reviews could have saved us from some front-end issues.
Then, there was the question of review culture. While we were mostly quite successful with it, we couldâ€™ve been a bit more stringent with the adoption at the early stages. Given that the team was formed from three organizations (with three different review backgrounds), we should have made everything clear from the get go. It would have been a step away from self-organization, but we would have set a meaningful default, then allowing the team to find its own path. At the very least, review completion should have been a part of the â€śdoneâ€ť criteria from a much earlier point of time in the project.
All in all, reviews made a big difference in the project. One more thing we couldâ€™ve done better is follow-up: We should have gone through common review comments more often and intentionally disseminate information about the issues on a weekly basis. We didnâ€™t have a process for that. In a bigger team and a more intense schedule, this would have been even more important.
Up next: Tools
Thatâ€™s it for the review methodology. The next post will be about the â€śhowâ€ť of reviews, including the tools (Crucible). At a later stage, I will return to the topic of reviews by explaining a few of the common issues we found and giving some ideas on how to tackle those issues.
What drove us to this?
When the concept of the site was born, high-fidelity user interface and visual appeal were high on the priority list. As you can see by browsing the site, particularly recipe search and articles, a hefty amount of work has been done to make the content look good. This includes contributions from artists, copywriters and photographers.
Once we had all this good-looking content, the challenge moved on to the viewing experience: How can we make it as enjoyable as possible for the user to browse the content? There had to be that certain feeling of abundance, and some of that comes from speed: new content has to be available so fast you think you can never consume it all.
But it wasnâ€™t just viewing the content, it was also about entering it. For example, the recipe editor. Few home cooks have written down exact instructions on how to prepare the dishes they have invented. Sure, many write down the ingredients, but itâ€™s a real effort to produce a concise set of instructions, in followable order, on how to reproduce the dish.
What kind of user interface would provide a reasonable approach to this? A set of text boxes would work, but be very clumsy in terms of reordering entered content. A textarea might be acceptable, but then dividing the instructions into clearly printable steps would involve splitting on line feeds or other semi-technical tricks, likely to mislead the user or produce unwanted results.
Client-side scripting shouldnâ€™t be your first option
Both examples above added up to this: We wanted faster search-based browsing, which called for a client-side implementation. And we wanted drag-and-drop rearrangeability for recipe entry, as other options would simply have been too unusable.
After all the debate, there was a valid case for a JS-driven user interface. It wasnâ€™t our first idea, nor was it an obvious choice to make. But in the end we did make that choice, and again a few more times along the way. Here are some of the aspects that affected our decisions:
- Why do we want this? (see above)
- If yes, would it involve a reasonable amount of work compared to the user benefit?
Different cases, different reasons
4) Features where we just didnâ€™t want to bother with the scriptless implementation
Finally, there are the features which could have been implemented without scripting but were not â€“ usually simply because we didnâ€™t have the time or considered something else more important. Perhaps the most visible example of this was the recipe scaling feature, allowing you to change the number of portions for any recipe.
Superficially, itâ€™s just two buttons that scale the recipe up or down. Behind the scenes, a whole lot of things happen when scaling. We use a JSON request to get the scaled recipe ingredients from the backend and then render it on the page using jQuery templating.
We could relatively easily add an HTTP GET based approach to scaling, providing links for +/- and then using querystring arguments for conveying the portion count. That might be something weâ€™re looking at later on.
- Increased complexity in terms of testing; itâ€™s very difficult to get all the happy and unhappy paths tested, both for JS and HTML.
All in all, there are quite a few hurdles that need to be considered. Accessibility is a very relevant and real question, but itâ€™s a business one. We wouldnâ€™t have that problem in an ideal world, but since few projects have the resources to actually deliver fully usable pure-HTML experiences, most will practically have to find a compromise.
While at it, I want to mention a couple of common reasons I hear for not using client-based technology.
- â€śWe canâ€™t make it work the same for everybodyâ€ť â€“ probably true, but itâ€™s also a cop out. Check with your business owner if thatâ€™s really what you need to do.
- â€śWe canâ€™t make it work the same across all the browsersâ€ť â€“ partially true, but if your understanding comes from times before jQuery et al., do refresh yourself. Getting CSS layout to work across browsers is much harder.