Heikniemi Hardcoded

Jouni Heikniemi's IT adventures

• Home
• About

Valio.fi deep dive #5: Content Management Systems as platforms

After a brief hiatus, it’s time to look at the Valio case again. This time, I’ll explain a few decisions behind our content management model. I’m sure some of this sounds familiar to almost every web site developer, although most won’t probably dive as deep as we did.

The setup

So you’re developing an ASP.NET MVC web application. You whip up Visual Studio and crank out a few controllers, views and whatnot. Ka-zoom, you have the working skeleton version available in a day or so. At this point, what you’re actually doing is that you’re exposing your key business objects – typically the database – through a server and a browser.

A few days or weeks later you ship the app, and immediately a business user tells you that “Actually, that segment of text is wrong. The correct one is…”. You quickly update a view. Two weeks and thirty updates later, you grow tired. Not only do they want to change the text, but they also want to swap in new images. And one day, you’ll get a mail asking you for a chance to slip in an additional information page. At a specified URI, of course. “Can I have it added to the navigation, too?”

At this point, you’re likely to have a few content delivery related customizations across your codebase. Your route table has a handful of entries specifically for campaign and info pages. Your version control history for the Views directory shows an increasingly high percentage of checkins related to one-off customization requests.

There will be a day when you find yourself asking: Should I just have started with a content management system (CMS), using extensibility hooks to implement your business functionality?

The problem

All web sites require some kind of CMS functionality: the ability to edit typical fragments of web content. Because of this, almost all web sites are based on a CMS. And there are so many of them; see the Wikipedia list.

At the other end of the spectrum, most web applications have very limited needs for content editing. Almost all have some requirements in this regard: even the most closed warehouse management tool often has a small area reserved for administrative announcements.

Most web projects fall between these two extremes. A typical e-commerce application is mostly an application, but it definitely needs content input: product descriptions, images, special offers etc. all need to be designed, tested and typed in on the go, without needing a developer’s input. A complex and diverse collection of content (such as Valio) is by definition a site and needs to be editable, but it also has a huge load of functionality – some of which may be considerably easier to implement using plain programming tools, not a CMS framework.

Picking our sides

When planning the technology strategy for the Valio project, we knew we were in trouble no matter what we picked.

There were requirements for considerable application-like functionalities including user-produced complex content (recipes), APIs for third parties, and demanding search rules. Simultaneously, we knew we would have to entertain a diverse group of content producers, from experienced web editors to amateur moderators.

If we chose a CMS… We would have at least a stub implementation for most of our CMS functionalities. We would implement all our logic using the CMS’s extensibility API. This might range from being moderately acceptable to extremely painful, but we probably wouldn’t know before we tried. We would have an authentication / user profile system available out of the box. However, most profile systems are not designed to be very well extendable. In particular, most don’t support complex integration to external directories.

If we wrote a custom application… We would have a relatively straightforward task implementing all the custom things. Hard things are still hard, but we’d be able to estimate them with reasonable confidence. We would have to write all the CMS tools by hand. Given our reasonably complex set of requirements (page previews, limited versioning, web part –like composability, customizable URIs, page templates), we knew this was going to be quite a few lines of code. We could do whatever we want with the authentication. Of course, that meant doing it all by hand, from square one.

 

As you probably know by now, we picked the custom application route. The CMS side had its perks, but we decided against it for the following reasons:

• The client had a fairly particular vision of the content management model. While they didn’t have stated requirements for the administration UI, we knew there were quite a few practical requirements, particularly regarding bulk moderations, that were not typically sufficiently implemented in CMS packages.
• While we had lots of CMS experience, we also had the requirement of using Microsoft technology, and an inner desire to use MVC to enable very fine-grained HTML output management. We also preferred an open source approach to ensure we could deliver everything we wanted. That left us in a situation where none of us had experience on a CMS that would match the requirements. Since evaluating CMS extensibility is very time-consuming, we didn’t want the additional risk of eating our precious days.
• On the other hand, having lots of CMS experience (including developing two of them) gave us a head start on designing the necessary infrastructure. Thus, we felt less intimidated by the challenge of creating our own tooling.

At the crossroads?

If you’re facing the same choice, I wouldn’t blindly recommend following us. We have been met with plenty of criticism and surprised faces when telling this story. Many people consider custom app writing a symptom of the NIH syndrome, particularly on a field as well established as CMSes. Also, it is a non-trivial exercise even if you have a good idea on what you’re doing.

The key lesson here is to play to your strengths, and choose by the project type. If you have a team with experience in a particular CMS and you have complex content management requirements, that particular CMS is likely to be a good idea. Then again, if all your users need is a single bulletin board for announcements, taking on a CMS framework is probably a hugely unnecessary piece of extra baggage.

However, one important thing is schedule predictability. If custom code and a CMS seem equally strong, consider any pre-baked system a risk in terms of change management: you quite likely cannot predict all its design constraints in beforehand.

For example, the requirements for Like button throttling in the Valio case were discovered reasonably late in the project, as was the moderation workflow for user's recipes. Most CMSes don’t offer smooth customizability in scenarios like this, and thus a small requirement can suddenly result in a larger refactoring, perhaps writing a module that replaces a part of the CMS itself. You would also be excessively optimistic in thinking that relatively obscure elements such as community content workflows would – or even could – be defined before the project.

The diagram below illustrates some of the design aspects and their weight in a totally customized scenario as well as a CMS-based one.

Not all CMS platforms are equal. The right end of the axis represents the versatile but hard-to-extend CMS solutions like SharePoint. The middle section of the chart represents CMS stacks that are more like toolkits for do-it-yourself site development; there are plenty of open source CMSes that are unfinished enough to be called such.

The conclusion

We picked what we picked because of 1) the requirements we had and 2) the people we were. Neither is irrelevant, and this is a key takeaway: A different team might pick an entirely different solution for the same requirements, and they might quite well be right.

You won’t have an easy time deciding on this: it’s a complex architectural choice, and estimating the actual impact of either option is reasonably hard. In the next post, I’ll discuss the key technical choices of our CMS implementation (on class/table level) to give you an idea on what sort of challenges you might be facing and help you in the process of gauging your cliff.

October 20, 2011 · Jouni Heikniemi · 3 Comments
Tags: CMS, valio.fi · Posted in: Web

Looking back at TechDays Finland 2011

Pretty close to half a year ago, Microsoft held the largest annual developer + IT Pro event in Finland, TechDays 2011. In six more months, it’s happening again.

As I was considering the various topics I might talk about, it always took me back to thinking about the years I’ve been talking there. What did people like? What kind of topic would interest people? How can I be better? My talks generally seem to fill a room of 100-200 people, but what do I have to say that’s worth 100-200 hours of Finnish developers’ time?

The TechDays speakers have had almost nonexistent visibility at the feedback gathered from the event. Well, that is, until now. Allow me to present the TechDays 2011 Feedback Infographic: (click on the image for additional resolution)

 

Challenges for TechDays 2012 speakers

There are plenty of conclusions one can draw from the data. The one that struck me the most is that people want highly practical information, but not without the theory. Case presentations beat pure theory hands-down, but the most popular sessions were the ones with high-energy presenters, a sound theoretical basis and a continuous pummeling of practical demos.

As I’m left pondering this, I want to offer three TD2012 challenges for fellow Finnish speakers:

• Sessions in Finnish scored 0.27 (on a scale of 1..5, that’s a lot) lower than sessions in English. Many attendees seem to gravitate towards the heavily rehearsed tracks coming from foreign travelling speakers. This must stop. Even though I think many of the travellers are absolutely great, Finnish technology professionals must be able to be more relevant and interesting*.
• Developer sessions scored a 3.61, which isn’t particularly bad, but it’s not good either. Many developer sessions are far too much based on the equivalent PDC/TechEd/Build sessions, perhaps even slides. Let me repeat: people want practical information and your experience with the theory. Let’s do better this year.
• Read Scott Berkun’s Confessions of a Public Speaker and either Presentation Zen or Slideology. Reduce the amount of verbiage in your decks and talks, and replace it with raw energy.

*) While berating the state of Finnish speaking, I must tip my hat off to Sami Laiho, a local Windows MVP. As stated in the infographic, the man did five different presentations, scored a 4.43 average (which would, had Sami’s presentations been a single track, been the most popular track of all TechDays), and pulled off the most popular presentation with a stunning 4.58 score. Oh, and the 4.58 was done on day 2 first slot, which speakers typically shun as the previous night’s attendee party is considered to hamper the contact with the audience. Blah blah.

Data disclaimer

A few words on the data used:

This post is based on a data dump of TechDays 2011 feedback I have received from Microsoft. No personally identifiable information was ever transmitted to me. The infographic has been cleared for publication by Microsoft, but such publication probably indicates no endorsement or broader approval. I do not have permission to redistribute the raw data, so questions for further info may or may not get addressed.

This blog post contains conclusions and opinions, which are naturally mine. On the other hand, the infographic is based on objective, raw event data with the following exceptions:

• Some questions have been grouped together for better illustration (namely, the food and venue/organization ones).
• I have manually divided the presentations into “type” categories (theory, practice, case, lecture series). This grouping is non-official.
• Grouping presenters was done by me. Deeming people “professional speakers” or “nobodies” involved subjective analysis.
• The textual description and the visualization is mine. The original data sheet nominates no kings, nor were pizza slices actually available.

Thanks for listening – and let’s hear your thoughts on TechDays 2011 and its feedback :-)

October 10, 2011 · Jouni Heikniemi · 26 Comments
Tags: infographics, TechDays · Posted in: General

9 things about one Offbeat year

One year and a couple of days ago we kicked off Offbeat Solutions, our consulting company of four people. We are still the same four, but far bigger four: we know more, we’ve done more, we’ve seen more. Here are a few insights from the first year.

3 things we nailed

Customer satisfaction. We took deliberate risks in picking our projects, but didn’t overdo it. We managed to deliver challenging things with a positive attitude, and people around us seem quite happy. We wanted to become trusted advisors to our clients, and for most, we did.

Not hiring anyone. It would have been easy (and even profitable) to get new people along; we had exactly the right cases for that, and we could have done so with acceptable costs. But had we done that, we wouldn’t have this strong unity, the ability to take on future incertainty with extreme trust in ourselves. There will be time for new hires, but it is not yet.

Setting up the office late enough. We discussed this office thing early on, but executed right at the 12-month mark. And damn, am I happy about it. Having worked together for a year, we had a reasonably good idea on what we’ll need to be productive (and have fun). Had we gone for the place instantly, we would not have had the resources to make it good. We probably still wouldn’t have moved out, simply because of the effort it takes. We could’ve been cheaper on this, but probably not much more efficient. More on the office thing in a later post.

3 things we sucked at

Teamwork rigor. This may come as a surprise to you, particularly if you worked with us. Still, internally, we feel that we haven’t been at the top of our game in working together. Sure, we beat the average organization hands down, but we find ourselves insufficiently seeking support from our each other – even when we obviously have a great chance. One of our 2011-2012 goals is to kill all procrastination by not only allowing, but demanding requests of assistance when things seem to slow down.

Technology breadth. We wanted to focus in working with the Microsoft stack, but take in the best parts from other technology families. Sure, we leverage lots of open source and external tools, but we’re nowhere near good enough. We write great ASP.NET MVC apps, but after our second year, we should have zero problem writing the same with Node.js, Rails, Akka or whatnot. And even inside the Microsoft stack, there’s plenty to learn.

Sharing with the community. We had the vision of being a storyteller, a company whose experiences would be publicly shared. We did some of that (first and foremost, our posts on valio.fi), but we were nowhere near as loud as we wanted to be. Given that we consider the year a great learning experience, it’s simply a shame that we haven’t found the time to share more.

3 things that made the year worth living

Work/life balance. Can’t say it was perfect, and we certainly worked too much at one time. Still, we have exploited our freedom of choosing where and when we work. There were frustrations, but few of them were really caused by too many hours.

Sufficient diversity. We did quite a few things. We worked on CRM strategies and security reviews. We wrote code and helped manage projects. We tackled SharePoint, Azure and PHP all alike. We worked with startups, SMBs, large enterprises, NGOs and the government. All great learning experiences.

The fun of it. We developed a culture – of course, not really intentionally. We developed a (bad) taste of office music, we printed out big pictures of legendary artists on the walls – and naturally, we started wearing military uniforms made of rubber during code reviews. We were supposed to be offbeat, right?

Summary

It’s been a blast, but we’ve grown rather critical about ourselves. Our start was way smoother than we expected, and that gave us a somewhat false feeling of security.

We’ve been fighting it for months now. It would be easy to bury ourselves into a few big customers, learn to understand their business deeply and become irreplaceable. But then, a few years down the line, we’d find ourselves fat, lazy and unlearned. Pretty much the opposite we planned originally.

Plan for 2011-2012: Work on the most difficult and ugly things we can find. Win ourselves every day. We’ll be back to tell you how we’re doing.

September 2, 2011 · Jouni Heikniemi · 2 Comments
Posted in: Entrepreneurship

SANKO-tapahtuma: Funktionaalinen ohjelmointi ja F#

This post is an exception from the English majority of my writings. I will be summarizing the Finnish .NET User Group’s event on F# and functional programming, and will do it in Finnish. In summary, a great event!

Neljäs SANKO-tapahtuma järjestettiin 9.6. ohjelmistoyhtiö Reaktorin tiloissa Helsingin keskustassa. Tuomas Hietanen ja Rami Karjalainen alustivat kertomalla kokemuksiaan funktionaalisesta ohjelmoinnista ja esittelemällä F#-kieltä. Kolmen tunnin tiukan esityksen jälkeen ilta jatkui ruokailulla ja saunomisella. Paikalla oli kokonaisuutena kolmisenkymmentä ihmistä.

Tuomaksen ja Ramin esitykset voisi tiivistää seuraavasti: He näkivät funktionaalisuuden olennaisena positiivisena trendinä, ja kokivat F#:n erittäin vakavasti otettavana kielenä vähintäänkin liiketoimintalogiikan ja datan toteutukseen. He olivat pitkään tehneet yhdessä vakuutusalan hanketta, jossa F# oli keskeisessä roolissa.

Yleisössä oli vahvaa kiinnostusta aiheeseen, ja Tuomas saikin selittää funktioteoriaa ja muita periaatteita varsin pitkällisesti. Käyttökokemus F#:sta itsestään oli harvassa, mutta C#:lla funktionaalista ohjelmointia – siis LINQ:n tehokäyttöä – oli kyllä harrastanut monikin. Funktionaalisuutta harjoitellessa F# nousi C#:a suosituimmaksi mm. vahvemman tyyppi-inferenssin (tyyppien automaattinen päättely, jolloin koodissa tarvitaan vähemmän turhaa tyyppien toistelua ja generics-kulmasulkeita) ja interaktiivisuuden takia. Käytiinpä esitysten lomassa myös keskustelu, jossa pohdittiin testivetoisen kehittämisen (TDD) arvoa ja sitä, onko itseasiassa interaktiiviseen REPL-työkaluun pohjautuva interaktiivinen kehitysmalli tehokkaampi.

Tuomaksen esitys: kalvot ja esimerkkisovellus

Ramin matskut: kalvot ja koodi

SANKO-toimintaan voit tutustua Facebookissa ja LinkedInissä. F#:lle on myös oma käyttäjäryhmänsä.

Kiitos kaikille osallistujille ja esiintyjille – ja tietysti myös Reaktorille sponsoroinnista. Ja hei, Reaktorin läsnäolo .NET-kentällä tarkoittaa siis sitä, että Suomen parhaana työpaikkana monesti palkittu softatalo on nyt auki myös Microsoft-osaajille. Se on merkittävä asia, enkä sano tätä vain siksi, että sain hyvää ruokaa. :-)

June 26, 2011 · Jouni Heikniemi · 169 Comments
Tags: F#, SANKO · Posted in: General

Valio.fi deep dive #4: Solving the ORM dilemma

In my series on valio.fi deep dives, I’ll return to the original fields of Offbeat and discuss some backend solutions. We won’t dive into the code today, but will take a look at some hard architectural choices.

Essentials of your ORM decision

We chose to build a reasonably large web site using NHibernate. A few members of the Microsoft developer crowd immediately asked: “Why not Entity Framework?” Meanwhile, a few of our developer friends had a lengthy discussion deeming ORMs worthless and unnecessarily complex.

There are really two subquestions here:

1. Should we use OR mapping to make data access more straightforward?
2. If yes, which OR mapper should we use?

Let me be absolutely clear on this one: I think both those questions are important to consider honestly. Many object-oriented developers feel that automated object/relation mapping is an essential part of developer productivity. I say that’s a valid point, but somewhat self-supported: ORM is important if you aren’t good enough at performing data access without it.

In this post, I’ll go over our decision making process, and then look back at where our use of NHibernate succeeded and failed. Mind you, the preceding emphasis is not a style issue: We take full responsibility for both our successes and failures, and do not intend to blame or praise NHibernate unnecessarily. It was just lying around, we used it.

To ORM or not to ORM?

This question was at the top of the list a few years ago. Now ORMs are mature enough to be the default choice for most developers, thus eliminating the question. Although the option of working without an ORM was debated, we sort of wanted one anyway. Our decision was based on a few facts:

1. We had a non-trivial (although not particularly complex by any measure) business model, consisting of a few dozen entities.
2. We had reasonably straightforward query requirements, mostly based on foreign key navigation.
3. We had a mostly-read model, which ORM could naturally support by providing features such as identity mapping and caching.
4. We had relatively few complex write operations and thus thought we’d get away without encountering the worst parts of mappings.

All these are good signs for ORM use. We weren’t entirely right in our predictions when making these decisions, but looking at it afterwards, I’m not sure we could have done better.

Picking the titans to fight

The next step was to compare Entity Framework to NHibernate.

Wait, why exactly these two? Welll, there are two factors: First, we had NHibernate experience available. Second, EF was the Microsoft Option. Well, we also had experience in other ORMs beside NHibernate and EF, but after considering our maturity and expertise in each, decided to limit ourselves to these two.

“The Microsoft Option”? It’s usually the official platform-integrated benchmark solution. Razor for view engines, EF for ORM, WCF for REST implementations and so on. Often not the best option, but the best known one. Should always be one option when picking a technology – being well-known is a big bonus, plus your software design never passes even a cursory review without you being questioned about ignoring the default option.

Don’t confuse that with picking the Microsoft Option blindly. We as a community of Microsoft developers are plagued by a disease that encourages choosing the MSFT solution every time, regardless of factual benefits. That is just as bad as ignoring the Microsoft option without thought.

Being included in the platform is a merit that justifies being shortlisted into any technology comparison, but definitely not winning them.

The fact that we chose only those two products for comparison seems arbitrary. This is an important point. It is entirely possible that there could have been a better solution we just didn’t know of. We decided to limit ourselves to these two because we considered the cost of exploring other options greater than the potential benefit. For any given technology, the benefit of evaluating a new option is about this:

B = p * ( AE – LC ) – EC

In the equation, the benefit B comes from subtracting the learning cost LC from the added efficiency AE and multiplying the result with p, the probability of this technology actually being picked. Furthermore, you need to subtract the evaluation cost EC from the benefit (although conceivably EC and LC would somehow merge, but we’ll keep it simple now).

Now, consider this: For an OR mapper, sufficient evaluation of an unknown variant takes at least a day. The trivial stuff you do in an hour, but going through all the complex scenarios (custom field types, xml handling, transactions, caching, cascading deletes, …) takes at least a day. The learning cost of doing all that properly is at least 5-10 days, particularly in a team of multiple developers. Even if we considered that some Acme ORM would be a contestant equally favorable to EF and NHibernate (i.e. p = 1/3), we would get:

B = 1 / 3 * (AE – 5) – 1

Quickly looking at it, we realize that with AE < 8, we would probably gain no benefit at all. Even if we estimated (before evaluating anything!) that a new technology saved us a whole month (20 man-days) of effort, our estimated benefit would be 1/3 * (20-5) –1 = 4 days. Would four days actually offset spending the one day in evaluation? And if we chose that, we’d gain 15 days but take a whole bunch of risk in exchange. How well can future generations of maintenance programmers benefit from this choice? And mind you, 1 day for eval, 5 for learning and 1/3 for usage probability are very optimistic estimates.

I’m very much pro-evaluation when choosing technologies, but you have to know when to stop. Considering that we didn’t plan to do many man-months of data management effort anyway, the potential gains never felt important enough to warrant the exploration effort. Thus, we focused on things that we found more productive.

The discussion above has nothing to do with ORMs per se, but ORMs are a good demonstration of this principle: since each stack does roughly the same things, knowing one of them relatively well is an extremely strong argument for picking it up and getting to real work.

Entity Framework vs. NHibernate, real arguments

We chose NHibernate for a few reasons. The key arguments were:

• NHibernate is more mature (and at the time the decisions were made in September 2010, the difference was even greater)
• NHibernate was better equipped for distributed caching (EFCachingProvider labels itself as a “sample”) and manipulation of fetching strategies.
• NHibernate + FluentNHibernate provided a reasonably approachable way to configure various complexities; EF’s approach of hiding some of this in a design diagram isn’t entirely without problems.

We did give EF serious consideration – finding people with NHibernate competence isn’t trivial, and as such, NHibernate is a hindrance to the further development. However, given the constraints we had, we had more confidence in choosing technology with a longer history (particularly on the Java side).

Regrets?

Our data layer architect Lauri* spent considerable time tweaking and tuning the data access mechanics – but most of this was done to create a smooth way to organize DAL code and make it accessible through the IoC mechanism we had. The bulk of that work was unrelated to NHibernate in itself, though much of it would have been different if we had opted for raw data access instead.

Most significant architectural headaches we had with NHibernate involved some of the more complicated structures. We have an everything-in-one-table –modeled class hierarchy with a discriminator field and an XML column for subclass data storage. Getting this to work wasn’t exactly smooth, nor was parsing the same column into two distinct fields. I may get back to this in more detail if there’s interest.

Performance problems are one of the most common fears with ORMs. For us, query performance wasn’t really a problem – reckless use of lazy loading did result in some issues, but this is more a result of underconstrained data usage than a problem related to choice of ORM itself. We have a few scenarios where NHibernate’s query cache manipulations actually slow down some operations, and we had to work around that. Most of the time, NHibernate’s fairly intuitive system of specifying fetching strategies worked like a charm.

Perhaps the most head scratching was caused by cascading delete/updates and ordered lists with index values in the database. Each of those required special trickery and effort, and particularly complex update scenarios were considerably difficult to verify as working correctly. These are the definite drawbacks in using an ORM – the abstraction really gets in your way far more than it benefits. Of course, such extremes do not typically occur in the vast majority of the program code.

*) We never appointed a data layer architect. He just grew up to that role. We like it that way.

Lessons learned

NHibernate did a good job, but it did exhibit all the problems of an ORM. Man, that stuff is hard! I mean, both EF and NHibernate make it very easy to whip up the simple demo application. If you have a reasonably straightforward business application with few concurrency issues and low performance requirements, chances are an ORM will totally save you. If you have a high-volume application with severe execution time constraints, you will probably encounter details that you never wanted to hear about.

I think the cost of using direct data access (data readers, commands and the like) is relatively linear compared to the project complexity. ORMs are much easier for a long time, but once you get to the hard end of the scale, their intricacies really start hurting you. I think this is particularly true for Entity Framework; for NHibernate, the availability of the source code provides some relief – or just just an extra measure of confidence – for the most extreme hacking scenarios.

Don’t pick an ORM to make things easier unless you’re dead certain that you’ll stay in the comfort zone of your competence. Of course, if you’re the resident EF/NHibernate surgeon, you’ll enjoy the benefits far further than a team full of newbies.

Personally, I wouldn’t change our basic decision to use NHibernate. However, I would be more judicious about our options in each particular point of the application – using an ORM somewhere does not necessarily mean it is the best thing to do everywhere. Also, I would make sure the ORM understanding is better disseminated inside the team – although not everybody needs to be able to fine-tune those fetch optimizations and lazy loads, everybody should understand the issues clearly enough to spot possible problems in reviews and other pre-deploy phases.

 

Up next, a week of vacation. Later in June, we’ll look into the back-end in more detail.

May 27, 2011 · Jouni Heikniemi · One Comment
Tags: data access, Entity Framework, NHibernate, ORM, valio.fi · Posted in: .NET

Valio.fi featured on the Vierityspalkki blog

A step aside from the deep dives!

The valio.fi project is now also presented in the famous Finnish Vierityspalkki.fi blog. If you can read Finnish, check out my guest post detailing the site and project from a general web developer (non-backend) perspective.

Meanwhile, deep dive #4 is cooking. The topic will be OR mapping, and I’ll post it on the next week. My apologies for the delay inbetween – you wouldn’t want the details (dental surgery and all).

May 20, 2011 · Jouni Heikniemi · 5 Comments
Tags: valio.fi · Posted in: Web

Valio.fi deep dive #3: Review tooling

After my previous post on review policy, let’s have a look at the tools we used for reviewing code.

I have found there to be two approaches to reviewing code. Let’s do a quick comparison first and discuss the tools of the trade next.

Patch-based review

Social review

The key concept here is that a developer prepares a set of changes, and publishes it for review. The reviewer focuses on the changes (i.e. “the patch”).  Reviews typically happen through a system (from an work-item attached unidiff file to a full web-based system). Comments on the review are often formalized documents, email messages or if a specialized system is used, its comment entries.

The developer presents the code to the reviewer. Some do this in a meeting room where multiple people review simultaneously, most of the time it happens pairwise at the dev’s workstation.  Reviews tend to focus on code understanding as a whole, and discussion about the code happens in-review. Written documents are sometimes produced, but most organizations rely on verbal communication and the developer’s personal notes.

 

Mind you, this is not review theory, but a very important split when looking at the tools. Most projects can easily benefit for the social approach because it requires less tooling and technical sophistication. My experience is that neither catches everything, and the ideal approach might involve both – but reviewing everything twice is often too expensive in terms of time.

We did both, and the social one was easy

Social reviews happen naturally when a team communicates well. The rigidity of the team organization usually determines if social reviews are specifically scheduled, if comments and their responses etc. are tracked, what kind of audit trail is required to accept a social review from a project management perspective and so on.

We used a reasonably liberal system. We required reviews (see the previous post), but accepted socials just like more formal, patch-based reviews. Either form of review was sufficient as long as the reviewer and developer felt like it. And we had no tools for social reviews. We had no document templates, no scheduling, no way to track the comments.

It worked for a reasonably well-bonded team of nine, and it might work for you. Don’t use lack of tooling as an excuse to not review socially; if tools are necessary for you, you’ll see the need to find them as you go.

Patch-based is a different beast

When you adopt patch-based reviewing, pay attention to the tools. I’ve done this for years with the pure unix diff/patch approach, and would not recommend it for development today. There are tools available, and looking at them is a good idea™. We used Crucible, but let me be crystal clear: We are tool pragmatists. We used Crucible because it was easily available, relatively cheap and worked for us. We don’t claim it was the best solution even for us, much less you. If you have the liberty of choosing your tools, do check it out – but also look at the competition.

But before you go on a spree of tool comparison, let me remind you that your review abilities, processes and social capacity far overwhelm your selection of tools in terms of determining your success with code reviews. If you’re setting up a review system from the scratch, you might spend a day on finding the appropriate tooling, but spend four more thinking about what, how and why are you reviewing.

Enough with the intro, go Crucible

The rest of the post will now discuss our patch-based reviews and the use of Crucible (to which the repository browser FishEye is tightly integrated to). First, I’ll walk you through the typical elements of a review, then discuss some of the tooling experiences.

As discussed before, we were in the post-checkin camp. Therefore, our reviews started with the developer committing the code to our version control (Subversion). We had Crucible and FishEye set up to poll the repository on regular intervals. Thus, on our Crucibile web site, we continously saw the stream of checkins listed.

For any of the checkins, one could create a review and pick the reviewers desired. There are multiple ways to configure and use Crucible; we used an approach that forced the author to pick the reviewers, but the reviewers could ask for more people to join. Although we encouraged an open review culture where anyone could review anything, we felt it necessary to identify the reviewers responsible – this would give everybody a clear idea on what’s on their plate (“Oh, I have these four things to review today”).

In Crucible, each review request contains the files to be reviewed. Since Crucible is integrated to the version control, these files are picked right from the source control tree, and can be either diffs (changes between two versions) or whole files. In any case, the files are shown in the browser, with coloring for the diff elements.

The whole point of reviewing is to get feedback on the code. Crucible’s feedback system is based on two concepts: comments focused on a certain code block (one or multiple lines) and comments on the whole review. The former is an approach to very specific feedback. Crucible also allows replying to the comments, thus generating nested comment/discussion trees.

Side note: As you can see from the screenshots, we had English as the code language but Finnish as the working language, and reviews were conducted in Finnish. We didn’t consider find this to be a problem. For us, a code comment is documentation for the future developer (potentially not knowing Finnish); a review comment is discussion between the team now (which is totally Finnish).

We make no recommendation on this – apart from keeping the commenting/reviewing bar as low as possible. Finns generally comment more eagerly in Finnish, but your mileage may vary.

The review process is fairly simple: Every requested reviewer goes through all the files, adds his comments and marks himself as complete. Once every reviewer has completed the review, the author goes through the comments, replies on them, fixes the code and finally closes the review. Rounds of discussion can ensue inbetween, but are often better handled outside Crucible (see below).

Experiences and best practices

Keep your reviews small. In fact, keep your checkins small. If your checkins are small, your reviews are too. Fix one aspect of code in a single checkin, and then review that. Even though Crucible encourages the model of picking a source control checkin and clicking “Create Review”, you can and should split checkins for review. I’ve written about this in detail back in 2004, so why repeat myself?

Review layers of trust or reuse separately. For example, if you develop a feature that needs new general use tools you implemented in the tools library, consider reviewing the tools separately. Separate review of the tools forces more focus on the general usability of the tools as the time-pressed reviewer doesn’t have the inclination to just look at how they function at the specific call site. However, you don’t necessarily want to take this too far; reviewing user interface and related data access in one go is probably a good idea.

Review unit tests. If your code has unit tests, include them in the review – and review them. In fact, quite a lot of attention should be paid to the tests, as they are a good indicator of various code issues. Reviewer should understand the tests thoroughly and then analyze their thoroughness – are all error paths reasonably tested? This approach provides an excellent way to spot bugs that you would very easily skip when just reading the actual implementation code.

Don’t make reviews your discussion board. Prefer short comments, debate face-to-face and then summarize the result in a short comment. Chatting through Crucible is inefficient, and while reviews can have considerable short-term documentary value, this value is not improved by including all discussion. Including the relevant results of a discussion provides the best balance.

Combine review methodologies to reduce pain. An author about to check in a batch of changes should consider ways to improve the review experience. Since diff-based reviewing is based on line-by-line change tracking, certain types of changes make diffs very hard to read. For example, renaming a member in project-wide use causes scattered diffs for hundreds, maybe thousands of code lines. Such a diff is very cumbersome to review properly (i.e. verify that nothing else has changed); it’s usually much better to do the rename together to eliminate the need for after-the-fact reviewing.

Don’t be afraid to take another spin. Every now and then, a review results in considerable code rewriting. This doesn’t mean the original code sucked; sometimes it’s just that the review reveals things that were not accounted for (a similar construct elsewhere in the project, a new requirement or whatever). If the code must significantly change because of the review, reviewers should be comfortable asking for another round (“make these changes and hit me with another review request”). Some may consider this unnecessary supervision and lack of trust for the code author, but they are the ones who misunderstand the key reasons behind reviews. Review until you have confidence in the code being fixed properly.

 

Up next: A dive into the backend technology, for a change.

May 4, 2011 · Jouni Heikniemi · 2 Comments
Tags: code review, valio.fi · Posted in: Misc. programming

Scott Guthrie moves on to lead the Azure Application Platform team

What could be important enough to interrupt the flow of posts on practical project experience? It is the reorganization that will throw every .NET developer’s favorite übergeek to a new position.

Yes, Scott Guthrie is leaving his post as the head honcho for many things such as IIS, ASP.NET, Silverlight and a slew of other products. His new task? Make sure the developers love Azure and get a solid development story on Microsoft’s cloud platform.

What does this mean?

• ScottGu’s blog will probably focus more on things related to cloud development (including private cloud). The division he’ll be working in delivers database services, BizTalk, AppFabric, WCF and workflows.
• There will have to be new champions for his previous products. Perhaps not directly in the terms of organizational position, but in the terms of being the public face for said products. Scott Hanselman and others will probably come to the rescue web-wise.
• Somebody else will probably keynote MIX12. Then again, PDC11 will probably see even a more empowered ScottGu preaching the Azure religion.

One thing I’m a bit worried about is the openness development that has been set well into pace in Guthrie’s previous organization. Co-operation with the jQuery team, integration of new open source technologies into the Visual Studio / .NET mix, growing support for releasing previously closed source, very open and straightforward relation to the developer scene… All this has been highly valuable.

I’m sure the Gu will take much of the goodness to his new org, but I’m less certain if the goodness will remain in the old one. At the same time, all this makes sense: ASP.NET and Silverlight are very, very mature compared to the development story on Azure. It’ll be interesting to see his impact over the coming months. Also, given the somewhat inherently closed nature of the cloud, the next steps of the openness story are worth looking forward to.

May 3, 2011 · Jouni Heikniemi · No Comments
Tags: ASP.NET, Azure, Silverlight · Posted in: .NET, Cloud

Valio.fi deep dive #2: Code review policy

In this second episode of my postings on valio.fi, I’ll look deeper into the concept of code reviews in the project.

Pre-post note: No, I’m not following a logical order with these. Rather, I’ll try to answer the asked questions as soon as I can while still maintaining a readable approach to the project details. Front-end posts will be interleaved with back-end magic, and there will be a dose of project management sauce inbetween. If you want to hear about something, ask for it in the comments.

In a project with numerous developers – in our case a scrum team of nine – the question of spreading code knowledge and ensuring certain quality bar tends to arise. There are numerous ways to tackle this, but we picked two principal approaches: Shared physical working space and code reviews.

Peer reviews vs. architect reviews

There are basically two alternative approaches to picking the reviewer. Either you have a reviewer/architect role (which may be rotating, but often tends to stick), or you can have a system of free peer reviews, i.e. anyone can review anything as long as they feel sufficiently competent about it.

Wanting to emphasize the equality of team members, we naturally chose the peer review model. However, few teams have a totally balanced review graph. Usually some people end up reviewing more code than others. This is partially a function of seniority, but also dependent on character, preferences and random factors (such as knowledge of a particular tricky subject).

Ours wasn’t balanced either. A couple of persons ended up being a part of most of the reviews. Still, we found the system to be fairly optimal: it did distribute a lot of knowledge, everyone had their code reviewed by several others and everyone reviewed somebody’s code.

Review guidelines

As with any project with more than a few people and a duration of months or more, writing down the intended methodology felt wise. Thus, we did have written review guidelines, although they were just that – common sense was allowed to rule over these idealistic methodology statements. Here is a summary of our guidelines with some examples and clarifications in parentheses:

Some notes on the guidelines

These rules, as written above, were applied at the start of the project. During the final few sprints, we relaxed the guidelines and focused on shipping.

No, wait, maybe that was what you expected to hear. Actually, we tightened the rules a bit for the final three sprints (six weeks) or so. Mainly, we merged the two first headings; essentially, we started to require reviews on all non-trivial changes to the code base. We also pushed this to our project management system; a sprint backlog item was not marked complete until it had passed review (for whatever definition of “passed”; we still allowed the author to decide whether review comments warranted code changes or not).

As you can see, we relied a lot on code authors. We allowed everybody to request review from anyone, meaning that we didn’t – nor did the project management – get involved in how much time was spent in reviews. And quite some time was spent, although we didn’t log any specific numbers. Personally, I spent almost half of my time reviewing code towards the end of the project, although admittedly I was one of the most active reviewers.

Forced vs. recommended: Before or after checkin?

There are basically two stages when you can do a review. Many open source projects take a fairly strict path: Only a select few people can commit or authorize commits to the source control, thus essentially creating a tight net for reviews. Often, in the largest projects, nobody commits anything without somebody else looking at it first. This makes sense when nobody can know the code authors.  The other alternative is to allow free commits, but review before releases or other milestones.

We were totally in the post-checkin camp. Everyone could commit anything, and we had no system in place to track if people actually did request reviews (until late in the game when we made it part of the completion criteria).

If you have the right culture, reviews are free

So yeah, we had a very author-controlled approach to reviews. But we also had a very open code quality culture, and it was not uncommon to question the validity of somebody’s approach over lunch. The debates could get quite heated, and we often spent a lot of time on issues that didn’t, in the end, have enough weight to warrant such use of time.

However, it all served a purpose. Our somewhat excessive quality discussions created a culture where writing bad code – or skipping reviews – wasn’t particularly lucrative. You never got publicly lambasted for a stupid bug in unreviewed code, but everybody felt better collectively owning code that had passed a review: a bug in well-reviewed code was a team mistake.

The reviews definitely had a time cost. Their positive impact is very hard to measure, given that we couldn’t know what the amount of bugs would have been without the review net. We caught some in the reviews, but probably prevented ten times more by conveying knowledge and setting examples during the review process.

A couple of weeks after shipping the site, many people kept wondering about the low amount of bugs discovered right after the launch. This sentiment, no matter how unmeasurable, is one of the key indicators of our success in terms of quality.

A perfect success?

Far from it. First of all, our front-end code wasn’t nearly as meticulously reviewed, for a variety of reasons (although the same attention to quality did apply, it wasn’t as collective). There were definitely scenarios where reviews could have saved us from some front-end issues.

Then, there was the question of review culture. While we were mostly quite successful with it, we could’ve been a bit more stringent with the adoption at the early stages. Given that the team was formed from three organizations (with three different review backgrounds), we should have made everything clear from the get go. It would have been a step away from self-organization, but we would have set a meaningful default, then allowing the team to find its own path. At the very least, review completion should have been a part of the “done” criteria from a much earlier point of time in the project.

All in all, reviews made a big difference in the project. One more thing we could’ve done better is follow-up: We should have gone through common review comments more often and intentionally disseminate information about the issues on a weekly basis. We didn’t have a process for that. In a bigger team and a more intense schedule, this would have been even more important.

Up next: Tools

That’s it for the review methodology. The next post will be about the “how” of reviews, including the tools (Crucible). At a later stage, I will return to the topic of reviews by explaining a few of the common issues we found and giving some ideas on how to tackle those issues.

April 28, 2011 · Jouni Heikniemi · 3 Comments
Tags: code review, valio.fi · Posted in: Misc. programming

Valio.fi deep dive #1: Understanding our JavaScript philosophy

In this first part of my series of postings on the Valio.fi project, I’ll discuss some design aspects of our use of JavaScript. We outlined these principles in the TechDays presentation, but the guidelines are worth repeating in print as well.

What drove us to this?

First, let us reiterate the project background and goals.

When the concept of the site was born, high-fidelity user interface and visual appeal were high on the priority list. As you can see by browsing the site, particularly recipe search and articles, a hefty amount of work has been done to make the content look good. This includes contributions from artists, copywriters and photographers.

Once we had all this good-looking content, the challenge moved on to the viewing experience: How can we make it as enjoyable as possible for the user to browse the content? There had to be that certain feeling of abundance, and some of that comes from speed: new content has to be available so fast you think you can never consume it all.

But it wasn’t just viewing the content, it was also about entering it. For example, the recipe editor. Few home cooks have written down exact instructions on how to prepare the dishes they have invented. Sure, many write down the ingredients, but it’s a real effort to produce a concise set of instructions, in followable order, on how to reproduce the dish.

What kind of user interface would provide a reasonable approach to this? A set of text boxes would work, but be very clumsy in terms of reordering entered content. A textarea might be acceptable, but then dividing the instructions into clearly printable steps would involve splitting on line feeds or other semi-technical tricks, likely to mislead the user or produce unwanted results.

Client-side scripting shouldn’t be your first option

Both examples above added up to this: We wanted faster search-based browsing, which called for a client-side implementation. And we wanted drag-and-drop rearrangeability for recipe entry, as other options would simply have been too unusable.

After all the debate, there was a valid case for a JS-driven user interface. It wasn’t our first idea, nor was it an obvious choice to make. But in the end we did make that choice, and again a few more times along the way. Here are some of the aspects that affected our decisions:

• Why do we want this? (see above)
• Would it be possible to implement a reasonably usable non-JavaScript version?
• If yes, would it involve a reasonable amount of work compared to the user benefit?
• What proportion of users would suffer from the lack of JavaScript support?

Different cases, different reasons

I’ll present a few use cases for JavaScript on the site and try to give an idea on the arguments behind the decisions.

1) Features where JavaScript support is completely optional, just providing a better UI

For example, login to the site. The login function is available in the footer bar – if you don’t have JavaScript, the login function just acts as a link to a login page with a standard HTML form. If JS is enabled, it renders a “toast” window, allowing you to log in within the context of the page.

This kind of use of JavaScript is trivial to defend: it harms nobody, so the only real effect is the effort of writing and maintaining it (and testing the non-JS version, which tends to be forgotten). The vast majority of the site’s JavaScript usage falls under this category; little tricks to make the site look better and work smoother.

2) Features where a reasonable JavaScriptless implementation is not doable

The recipe editor is a prime example here. While it would be possible to conceive a UI that would work without JavaScript, it would be a stretch to call it usable. The recipe editor requires a lot of complex data input and fairly sophisticated input guidance. For example, we know the available units of an ingredient only after the ingredient has been selected. The selection list is long enough to make <select> population  quite unwieldy (not to mention the UI of the selection itself!).

While there definitely are quite a few users without JavaScript support, what percentage of them would be likely to use JavaScriptless user agents to add recipes? We deemed this portion of users small enough to ignore. Sounds harsh, but it is, ultimately, a business decision.

Many projects fear these decisions, and thus paralyze themselves with JS improvements: “if we cannot build a complete, perfect graceful degradation story, we can’t use JavaScript at all”.

3) Features where the JavaScriptless implementation is vastly simpler in terms of features

Searches are the most prominent example here. We provide a decent search without JavaScript, but it’s far from the visual glory of the JS version. It is not even functionally equivalent. For example, the JS version allows multiple search criteria from the same category (for example, you could select to search for recipes with meat AND cheese), while the pure HTML version only enables searching by one option in each category.

Such restrictions were not always technically necessary. We could have created the search interface with lots of multi-select controls or massive checkbox groups, and thus replicate the user experience. However, we often found the simpler option more usable. For example, without JavaScript, we could not show real-time numbers of matches for each filter. Thus, the user would essentially be searching blindly, perhaps ending with zero results more often than desired.

4) Features where we just didn’t want to bother with the scriptless implementation

Finally, there are the features which could have been implemented without scripting but were not – usually simply because we didn’t have the time or considered something else more important. Perhaps the most visible example of this was the recipe scaling feature, allowing you to change the number of portions for any recipe.

Superficially, it’s just two buttons that scale the recipe up or down. Behind the scenes, a whole lot of things happen when scaling. We use a JSON request to get the scaled recipe ingredients from the backend and then render it on the page using jQuery templating.

We could relatively easily add an HTTP GET based approach to scaling, providing links for +/- and then using querystring arguments for conveying the portion count. That might be something we’re looking at later on.

TANSTAAFL

All the sweetness from JavaScript doesn’t come without it problems, though. In exchange for bypassing some of the HTML basic form controls, we hit the following issues:

• Accessibility degradation; impactwise ranging from mild (drag and drop without a mouse?) to severe (you just cannot scale a recipe without JavaScript support)
• More code means more bugs; to provide a JavaScript-based UI with pure HTML fallback often means coding some things almost twice.
• Increased complexity in terms of testing; it’s very difficult to get all the happy and unhappy paths tested, both for JS and HTML.
• Increased attack surface from a security standpoint; while JavaScript itself rarely presents a security hazard, the additional APIs must also be checked for security issues, DoS opportunities and whatnot.
• Considerably higher skill demands for team members; few backend developers understand the requirements for a decent JavaScript API, and while learnable on the go, it is expensive.

All in all, there are quite a few hurdles that need to be considered. Accessibility is a very relevant and real question, but it’s a business one. We wouldn’t have that problem in an ideal world, but since few projects have the resources to actually deliver fully usable pure-HTML experiences, most will practically have to find a compromise.

Unacceptable reasons for dissing JavaScript

While at it, I want to mention a couple of common reasons I hear for not using client-based technology.

• “We can’t make it work the same for everybody” – probably true, but it’s also a cop out. Check with your business owner if that’s really what you need to do.
• “We can’t make it work the same across all the browsers” – partially true, but if your understanding comes from times before jQuery et al., do refresh yourself. Getting CSS layout to work across browsers is much harder.
• “JavaScript is not a proper programming language” – this is a colored statement coming from somebody with a limited definition of “proper”, typically involving static typing and class-based inheritance. Rest assured, the expressiveness of JavaScript will not be the limiting factor in your project.
• “JavaScript doesn’t perform well enough” – True in a very specific sense, but an insufficient reason for complete dismissal of client-side scripting. There are scenarios where scripting performance can kill your site. Large DOM manipulations in particular can be a headache on older browsers. You need to test, test and test. But with some work and skill, you can do a lot even with IE 6 – and most uses of JavaScript will encounter no performance issues whatsoever.

Conclusion

I want to end the story with a concluding note: There is a lot of power in JavaScript. As usual, not everything works – or is easy to do. But if you are building a web solution where the user experience matters, you will be doing yourself a disservice if you didn’t even think about the possibilities provided by client-side programming on the web.

For a larger project, you would do well to devise a consistent, premeditated decision making framework for evaluating your JavaScript scenarios. Each instance of JavaScript use should provide reasonable benefit to offset the technical cost and possible accessibility issues. With that design-time check in place, client-side scripting can improve your site drastically.

April 18, 2011 · Jouni Heikniemi · 2 Comments
Tags: JavaScript, valio.fi · Posted in: Web

• « Older Entries
• Newer Entries »

• Categories

  • .NET
  • Bugzilla
  • Cloud
  • Entrepreneurship
  • General
  • Information Worker
  • Misc. programming
  • Web
  • Windows IT

• Archives

  • April 2016
  • January 2016
  • August 2015
  • April 2015
  • March 2015
  • February 2015
  • December 2013
  • June 2013
  • June 2012
  • May 2012
  • March 2012
  • January 2012
  • November 2011
  • October 2011
  • September 2011
  • June 2011
  • May 2011
  • April 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • December 2005
  • September 2005
  • August 2005
  • July 2005
  • June 2005
  • May 2005
  • April 2005
  • March 2005
  • February 2005
  • January 2005
  • December 2004
  • November 2004
  • October 2004
  • September 2004
  • August 2004
  • July 2004

• Meta

  • Log in
  • Entries RSS
  • Comments RSS