9 things about one Offbeat year

One year and a couple of days ago we kicked off Offbeat Solutions, our consulting company of four people. We are still the same four, but far bigger four: we know more, we’ve done more, we’ve seen more. Here are a few insights from the first year.

3 things we nailed

Customer satisfaction. We took deliberate risks in picking our projects, but didn’t overdo it. We managed to deliver challenging things with a positive attitude, and people around us seem quite happy. We wanted to become trusted advisors to our clients, and for most, we did.

Not hiring anyone. It would have been easy (and even profitable) to get new people along; we had exactly the right cases for that, and we could have done so with acceptable costs. But had we done that, we wouldn’t have this strong unity, the ability to take on future incertainty with extreme trust in ourselves. There will be time for new hires, but it is not yet.

imageSetting up the office late enough. We discussed this office thing early on, but executed right at the 12-month mark. And damn, am I happy about it. Having worked together for a year, we had a reasonably good idea on what we’ll need to be productive (and have fun). Had we gone for the place instantly, we would not have had the resources to make it good. We probably still wouldn’t have moved out, simply because of the effort it takes. We could’ve been cheaper on this, but probably not much more efficient. More on the office thing in a later post.

3 things we sucked at

Teamwork rigor. This may come as a surprise to you, particularly if you worked with us. Still, internally, we feel that we haven’t been at the top of our game in working together. Sure, we beat the average organization hands down, but we find ourselves insufficiently seeking support from our each other – even when we obviously have a great chance. One of our 2011-2012 goals is to kill all procrastination by not only allowing, but demanding requests of assistance when things seem to slow down.

Technology breadth. We wanted to focus in working with the Microsoft stack, but take in the best parts from other technology families. Sure, we leverage lots of open source and external tools, but we’re nowhere near good enough. We write great ASP.NET MVC apps, but after our second year, we should have zero problem writing the same with Node.js, Rails, Akka or whatnot. And even inside the Microsoft stack, there’s plenty to learn.

Sharing with the community. We had the vision of being a storyteller, a company whose experiences would be publicly shared. We did some of that (first and foremost, our posts on valio.fi), but we were nowhere near as loud as we wanted to be. Given that we consider the year a great learning experience, it’s simply a shame that we haven’t found the time to share more.

3 things that made the year worth living

imageWork/life balance. Can’t say it was perfect, and we certainly worked too much at one time. Still, we have exploited our freedom of choosing where and when we work. There were frustrations, but few of them were really caused by too many hours.

Sufficient diversity. We did quite a few things. We worked on CRM strategies and security reviews. We wrote code and helped manage projects. We tackled SharePoint, Azure and PHP all alike. We worked with startups, SMBs, large enterprises, NGOs and the government. All great learning experiences.

The fun of it. We developed a culture – of course, not really intentionally. We developed a (bad) taste of office music, we printed out big pictures of legendary artists on the walls – and naturally, we started wearing military uniforms made of rubber during code reviews. We were supposed to be offbeat, right?

Summary

It’s been a blast, but we’ve grown rather critical about ourselves. Our start was way smoother than we expected, and that gave us a somewhat false feeling of security.

We’ve been fighting it for months now. It would be easy to bury ourselves into a few big customers, learn to understand their business deeply and become irreplaceable. But then, a few years down the line, we’d find ourselves fat, lazy and unlearned. Pretty much the opposite we planned originally.

Plan for 2011-2012: Work on the most difficult and ugly things we can find. Win ourselves every day. We’ll be back to tell you how we’re doing.

September 2, 2011 · Jouni Heikniemi · One Comment
Posted in: Entrepreneurship

SANKO-tapahtuma: Funktionaalinen ohjelmointi ja F#

This post is an exception from the English majority of my writings. I will be summarizing the Finnish .NET User Group’s event on F# and functional programming, and will do it in Finnish. In summary, a great event!

Neljäs SANKO-tapahtuma järjestettiin 9.6. ohjelmistoyhtiö Reaktorin tiloissa Helsingin keskustassa. Tuomas Hietanen ja Rami Karjalainen alustivat kertomalla kokemuksiaan funktionaalisesta ohjelmoinnista ja esittelemällä F#-kieltä. Kolmen tunnin tiukan esityksen jälkeen ilta jatkui ruokailulla ja saunomisella. Paikalla oli kokonaisuutena kolmisenkymmentä ihmistä.

imageTuomaksen ja Ramin esitykset voisi tiivistää seuraavasti: He näkivät funktionaalisuuden olennaisena positiivisena trendinä, ja kokivat F#:n erittäin vakavasti otettavana kielenä vähintäänkin liiketoimintalogiikan ja datan toteutukseen. He olivat pitkään tehneet yhdessä vakuutusalan hanketta, jossa F# oli keskeisessä roolissa.

Yleisössä oli vahvaa kiinnostusta aiheeseen, ja Tuomas saikin selittää funktioteoriaa ja muita periaatteita varsin pitkällisesti. Käyttökokemus F#:sta itsestään oli harvassa, mutta C#:lla funktionaalista ohjelmointia – siis LINQ:n tehokäyttöä – oli kyllä harrastanut monikin. Funktionaalisuutta harjoitellessa F# nousi C#:a suosituimmaksi mm. vahvemman tyyppi-inferenssin (tyyppien automaattinen päättely, jolloin koodissa tarvitaan vähemmän turhaa tyyppien toistelua ja generics-kulmasulkeita) ja interaktiivisuuden takia. Käytiinpä esitysten lomassa myös keskustelu, jossa pohdittiin testivetoisen kehittämisen (TDD) arvoa ja sitä, onko itseasiassa interaktiiviseen REPL-työkaluun pohjautuva interaktiivinen kehitysmalli tehokkaampi.

Tuomaksen esitys: kalvot ja esimerkkisovellus

Ramin matskut: kalvot ja koodi

SANKO-toimintaan voit tutustua Facebookissa ja LinkedInissä. F#:lle on myös oma käyttäjäryhmänsä.

Kiitos kaikille osallistujille ja esiintyjille – ja tietysti myös Reaktorille sponsoroinnista. Ja hei, Reaktorin läsnäolo .NET-kentällä tarkoittaa siis sitä, että Suomen parhaana työpaikkana monesti palkittu softatalo on nyt auki myös Microsoft-osaajille. Se on merkittävä asia, enkä sano tätä vain siksi, että sain hyvää ruokaa. :-)

June 26, 2011 · Jouni Heikniemi · 165 Comments
Tags: ,  Â· Posted in: General

Valio.fi deep dive #4: Solving the ORM dilemma

In my series on valio.fi deep dives, I’ll return to the original fields of Offbeat and discuss some backend solutions. We won’t dive into the code today, but will take a look at some hard architectural choices.

Essentials of your ORM decision

We chose to build a reasonably large web site using NHibernate. A few members of the Microsoft developer crowd immediately asked: “Why not Entity Framework?” Meanwhile, a few of our developer friends had a lengthy discussion deeming ORMs worthless and unnecessarily complex.

There are really two subquestions here:

  1. Should we use OR mapping to make data access more straightforward?
  2. If yes, which OR mapper should we use?

Let me be absolutely clear on this one: I think both those questions are important to consider honestly. Many object-oriented developers feel that automated object/relation mapping is an essential part of developer productivity. I say that’s a valid point, but somewhat self-supported: ORM is important if you aren’t good enough at performing data access without it.

In this post, I’ll go over our decision making process, and then look back at where our use of NHibernate succeeded and failed. Mind you, the preceding emphasis is not a style issue: We take full responsibility for both our successes and failures, and do not intend to blame or praise NHibernate unnecessarily. It was just lying around, we used it.

To ORM or not to ORM?

This question was at the top of the list a few years ago. Now ORMs are mature enough to be the default choice for most developers, thus eliminating the question. Although the option of working without an ORM was debated, we sort of wanted one anyway. Our decision was based on a few facts:

  1. We had a non-trivial (although not particularly complex by any measure) business model, consisting of a few dozen entities.
  2. We had reasonably straightforward query requirements, mostly based on foreign key navigation.
  3. We had a mostly-read model, which ORM could naturally support by providing features such as identity mapping and caching.
  4. We had relatively few complex write operations and thus thought we’d get away without encountering the worst parts of mappings.

All these are good signs for ORM use. We weren’t entirely right in our predictions when making these decisions, but looking at it afterwards, I’m not sure we could have done better.

Picking the titans to fight

The next step was to compare Entity Framework to NHibernate.

Wait, why exactly these two? Welll, there are two factors: First, we had NHibernate experience available. Second, EF was the Microsoft Option. Well, we also had experience in other ORMs beside NHibernate and EF, but after considering our maturity and expertise in each, decided to limit ourselves to these two.

“The Microsoft Option”? It’s usually the official platform-integrated benchmark solution. Razor for view engines, EF for ORM, WCF for REST implementations and so on. Often not the best option, but the best known one. Should always be one option when picking a technology – being well-known is a big bonus, plus your software design never passes even a cursory review without you being questioned about ignoring the default option.

Don’t confuse that with picking the Microsoft Option blindly. We as a community of Microsoft developers are plagued by a disease that encourages choosing the MSFT solution every time, regardless of factual benefits. That is just as bad as ignoring the Microsoft option without thought.

Being included in the platform is a merit that justifies being shortlisted into any technology comparison, but definitely not winning them.

The fact that we chose only those two products for comparison seems arbitrary. This is an important point. It is entirely possible that there could have been a better solution we just didn’t know of. We decided to limit ourselves to these two because we considered the cost of exploring other options greater than the potential benefit. For any given technology, the benefit of evaluating a new option is about this:

B = p * ( AE – LC ) – EC

In the equation, the benefit B comes from subtracting the learning cost LC from the added efficiency AE and multiplying the result with p, the probability of this technology actually being picked. Furthermore, you need to subtract the evaluation cost EC from the benefit (although conceivably EC and LC would somehow merge, but we’ll keep it simple now).

Now, consider this: For an OR mapper, sufficient evaluation of an unknown variant takes at least a day. The trivial stuff you do in an hour, but going through all the complex scenarios (custom field types, xml handling, transactions, caching, cascading deletes, …) takes at least a day. The learning cost of doing all that properly is at least 5-10 days, particularly in a team of multiple developers. Even if we considered that some Acme ORM would be a contestant equally favorable to EF and NHibernate (i.e. p = 1/3), we would get:

B = 1 / 3 * (AE – 5) – 1

Quickly looking at it, we realize that with AE < 8, we would probably gain no benefit at all. Even if we estimated (before evaluating anything!) that a new technology saved us a whole month (20 man-days) of effort, our estimated benefit would be 1/3 * (20-5) –1 = 4 days. Would four days actually offset spending the one day in evaluation? And if we chose that, we’d gain 15 days but take a whole bunch of risk in exchange. How well can future generations of maintenance programmers benefit from this choice? And mind you, 1 day for eval, 5 for learning and 1/3 for usage probability are very optimistic estimates.

I’m very much pro-evaluation when choosing technologies, but you have to know when to stop. Considering that we didn’t plan to do many man-months of data management effort anyway, the potential gains never felt important enough to warrant the exploration effort. Thus, we focused on things that we found more productive.

The discussion above has nothing to do with ORMs per se, but ORMs are a good demonstration of this principle: since each stack does roughly the same things, knowing one of them relatively well is an extremely strong argument for picking it up and getting to real work.

Entity Framework vs. NHibernate, real arguments

We chose NHibernate for a few reasons. The key arguments were:

  • NHibernate is more mature (and at the time the decisions were made in September 2010, the difference was even greater)
  • NHibernate was better equipped for distributed caching (EFCachingProvider labels itself as a “sample”) and manipulation of fetching strategies.
  • NHibernate + FluentNHibernate provided a reasonably approachable way to configure various complexities; EF’s approach of hiding some of this in a design diagram isn’t entirely without problems.

We did give EF serious consideration – finding people with NHibernate competence isn’t trivial, and as such, NHibernate is a hindrance to the further development. However, given the constraints we had, we had more confidence in choosing technology with a longer history (particularly on the Java side).

Regrets?

Our data layer architect Lauri* spent considerable time tweaking and tuning the data access mechanics – but most of this was done to create a smooth way to organize DAL code and make it accessible through the IoC mechanism we had. The bulk of that work was unrelated to NHibernate in itself, though much of it would have been different if we had opted for raw data access instead.

Most significant architectural headaches we had with NHibernate involved some of the more complicated structures. We have an everything-in-one-table –modeled class hierarchy with a discriminator field and an XML column for subclass data storage. Getting this to work wasn’t exactly smooth, nor was parsing the same column into two distinct fields. I may get back to this in more detail if there’s interest.

Performance problems are one of the most common fears with ORMs. For us, query performance wasn’t really a problem – reckless use of lazy loading did result in some issues, but this is more a result of underconstrained data usage than a problem related to choice of ORM itself. We have a few scenarios where NHibernate’s query cache manipulations actually slow down some operations, and we had to work around that. Most of the time, NHibernate’s fairly intuitive system of specifying fetching strategies worked like a charm.

Perhaps the most head scratching was caused by cascading delete/updates and ordered lists with index values in the database. Each of those required special trickery and effort, and particularly complex update scenarios were considerably difficult to verify as working correctly. These are the definite drawbacks in using an ORM – the abstraction really gets in your way far more than it benefits. Of course, such extremes do not typically occur in the vast majority of the program code.

*) We never appointed a data layer architect. He just grew up to that role. We like it that way.

Lessons learned

NHibernate did a good job, but it did exhibit all the problems of an ORM. Man, that stuff is hard! I mean, both EF and NHibernate make it very easy to whip up the simple demo application. If you have a reasonably straightforward business application with few concurrency issues and low performance requirements, chances are an ORM will totally save you. If you have a high-volume application with severe execution time constraints, you will probably encounter details that you never wanted to hear about.

imageI think the cost of using direct data access (data readers, commands and the like) is relatively linear compared to the project complexity. ORMs are much easier for a long time, but once you get to the hard end of the scale, their intricacies really start hurting you. I think this is particularly true for Entity Framework; for NHibernate, the availability of the source code provides some relief – or just just an extra measure of confidence – for the most extreme hacking scenarios.

Don’t pick an ORM to make things easier unless you’re dead certain that you’ll stay in the comfort zone of your competence. Of course, if you’re the resident EF/NHibernate surgeon, you’ll enjoy the benefits far further than a team full of newbies.

Personally, I wouldn’t change our basic decision to use NHibernate. However, I would be more judicious about our options in each particular point of the application – using an ORM somewhere does not necessarily mean it is the best thing to do everywhere. Also, I would make sure the ORM understanding is better disseminated inside the team – although not everybody needs to be able to fine-tune those fetch optimizations and lazy loads, everybody should understand the issues clearly enough to spot possible problems in reviews and other pre-deploy phases.

 

Up next, a week of vacation. Later in June, we’ll look into the back-end in more detail.

May 27, 2011 · Jouni Heikniemi · One Comment
Tags: , , , ,  Â· Posted in: .NET

Valio.fi featured on the Vierityspalkki blog

A step aside from the deep dives!

The valio.fi project is now also presented in the famous Finnish Vierityspalkki.fi blog. If you can read Finnish, check out my guest post detailing the site and project from a general web developer (non-backend) perspective.

Meanwhile, deep dive #4 is cooking. The topic will be OR mapping, and I’ll post it on the next week. My apologies for the delay inbetween – you wouldn’t want the details (dental surgery and all).

May 20, 2011 · Jouni Heikniemi · No Comments
Tags:  Â· Posted in: Web

Valio.fi deep dive #3: Review tooling

After my previous post on review policy, let’s have a look at the tools we used for reviewing code.

I have found there to be two approaches to reviewing code. Let’s do a quick comparison first and discuss the tools of the trade next.

Patch-based review Social review
The key concept here is that a developer prepares a set of changes, and publishes it for review. The reviewer focuses on the changes (i.e. “the patch”). 

Reviews typically happen through a system (from an work-item attached unidiff file to a full web-based system).

Comments on the review are often formalized documents, email messages or if a specialized system is used, its comment entries.

The developer presents the code to the reviewer. Some do this in a meeting room where multiple people review simultaneously, most of the time it happens pairwise at the dev’s workstation. 

Reviews tend to focus on code understanding as a whole, and discussion about the code happens in-review. Written documents are sometimes produced, but most organizations rely on verbal communication and the developer’s personal notes.

 

Mind you, this is not review theory, but a very important split when looking at the tools. Most projects can easily benefit for the social approach because it requires less tooling and technical sophistication. My experience is that neither catches everything, and the ideal approach might involve both – but reviewing everything twice is often too expensive in terms of time.

We did both, and the social one was easy

imageSocial reviews happen naturally when a team communicates well. The rigidity of the team organization usually determines if social reviews are specifically scheduled, if comments and their responses etc. are tracked, what kind of audit trail is required to accept a social review from a project management perspective and so on.

We used a reasonably liberal system. We required reviews (see the previous post), but accepted socials just like more formal, patch-based reviews. Either form of review was sufficient as long as the reviewer and developer felt like it. And we had no tools for social reviews. We had no document templates, no scheduling, no way to track the comments.

It worked for a reasonably well-bonded team of nine, and it might work for you. Don’t use lack of tooling as an excuse to not review socially; if tools are necessary for you, you’ll see the need to find them as you go.

Patch-based is a different beast

When you adopt patch-based reviewing, pay attention to the tools. I’ve done this for years with the pure unix diff/patch approach, and would not recommend it for development today. There are tools available, and looking at them is a good idea™. We used Crucible, but let me be crystal clear: We are tool pragmatists. We used Crucible because it was easily available, relatively cheap and worked for us. We don’t claim it was the best solution even for us, much less you. If you have the liberty of choosing your tools, do check it out – but also look at the competition.

But before you go on a spree of tool comparison, let me remind you that your review abilities, processes and social capacity far overwhelm your selection of tools in terms of determining your success with code reviews. If you’re setting up a review system from the scratch, you might spend a day on finding the appropriate tooling, but spend four more thinking about what, how and why are you reviewing.

Enough with the intro, go Crucible

The rest of the post will now discuss our patch-based reviews and the use of Crucible (to which the repository browser FishEye is tightly integrated to). First, I’ll walk you through the typical elements of a review, then discuss some of the tooling experiences.

As discussed before, we were in the post-checkin camp. Therefore, our reviews started with the developer committing the code to our version control (Subversion). We had Crucible and FishEye set up to poll the repository on regular intervals. Thus, on our Crucibile web site, we continously saw the stream of checkins listed.

image

For any of the checkins, one could create a review and pick the reviewers desired. There are multiple ways to configure and use Crucible; we used an approach that forced the author to pick the reviewers, but the reviewers could ask for more people to join. Although we encouraged an open review culture where anyone could review anything, we felt it necessary to identify the reviewers responsible – this would give everybody a clear idea on what’s on their plate (“Oh, I have these four things to review today”).

image

In Crucible, each review request contains the files to be reviewed. Since Crucible is integrated to the version control, these files are picked right from the source control tree, and can be either diffs (changes between two versions) or whole files. In any case, the files are shown in the browser, with coloring for the diff elements.

image

The whole point of reviewing is to get feedback on the code. Crucible’s feedback system is based on two concepts: comments focused on a certain code block (one or multiple lines) and comments on the whole review. The former is an approach to very specific feedback. Crucible also allows replying to the comments, thus generating nested comment/discussion trees.

image

Side note: As you can see from the screenshots, we had English as the code language but Finnish as the working language, and reviews were conducted in Finnish. We didn’t consider find this to be a problem. For us, a code comment is documentation for the future developer (potentially not knowing Finnish); a review comment is discussion between the team now (which is totally Finnish).

We make no recommendation on this – apart from keeping the commenting/reviewing bar as low as possible. Finns generally comment more eagerly in Finnish, but your mileage may vary.

The review process is fairly simple: Every requested reviewer goes through all the files, adds his comments and marks himself as complete. Once every reviewer has completed the review, the author goes through the comments, replies on them, fixes the code and finally closes the review. Rounds of discussion can ensue inbetween, but are often better handled outside Crucible (see below).

Experiences and best practices

Keep your reviews small. In fact, keep your checkins small. If your checkins are small, your reviews are too. Fix one aspect of code in a single checkin, and then review that. Even though Crucible encourages the model of picking a source control checkin and clicking “Create Review”, you can and should split checkins for review. I’ve written about this in detail back in 2004, so why repeat myself?

Review layers of trust or reuse separately. For example, if you develop a feature that needs new general use tools you implemented in the tools library, consider reviewing the tools separately. Separate review of the tools forces more focus on the general usability of the tools as the time-pressed reviewer doesn’t have the inclination to just look at how they function at the specific call site. However, you don’t necessarily want to take this too far; reviewing user interface and related data access in one go is probably a good idea.

Review unit tests. If your code has unit tests, include them in the review – and review them. In fact, quite a lot of attention should be paid to the tests, as they are a good indicator of various code issues. Reviewer should understand the tests thoroughly and then analyze their thoroughness – are all error paths reasonably tested? This approach provides an excellent way to spot bugs that you would very easily skip when just reading the actual implementation code.

Don’t make reviews your discussion board. Prefer short comments, debate face-to-face and then summarize the result in a short comment. Chatting through Crucible is inefficient, and while reviews can have considerable short-term documentary value, this value is not improved by including all discussion. Including the relevant results of a discussion provides the best balance.

Combine review methodologies to reduce pain. An author about to check in a batch of changes should consider ways to improve the review experience. Since diff-based reviewing is based on line-by-line change tracking, certain types of changes make diffs very hard to read. For example, renaming a member in project-wide use causes scattered diffs for hundreds, maybe thousands of code lines. Such a diff is very cumbersome to review properly (i.e. verify that nothing else has changed); it’s usually much better to do the rename together to eliminate the need for after-the-fact reviewing.

Don’t be afraid to take another spin. Every now and then, a review results in considerable code rewriting. This doesn’t mean the original code sucked; sometimes it’s just that the review reveals things that were not accounted for (a similar construct elsewhere in the project, a new requirement or whatever). If the code must significantly change because of the review, reviewers should be comfortable asking for another round (“make these changes and hit me with another review request”). Some may consider this unnecessary supervision and lack of trust for the code author, but they are the ones who misunderstand the key reasons behind reviews. Review until you have confidence in the code being fixed properly.

 

Up next: A dive into the backend technology, for a change.

May 4, 2011 · Jouni Heikniemi · 2 Comments
Tags: ,  Â· Posted in: Misc. programming

Scott Guthrie moves on to lead the Azure Application Platform team

imageWhat could be important enough to interrupt the flow of posts on practical project experience? It is the reorganization that will throw every .NET developer’s favorite übergeek to a new position.

Yes, Scott Guthrie is leaving his post as the head honcho for many things such as IIS, ASP.NET, Silverlight and a slew of other products. His new task? Make sure the developers love Azure and get a solid development story on Microsoft’s cloud platform.

What does this mean?

  • ScottGu’s blog will probably focus more on things related to cloud development (including private cloud). The division he’ll be working in delivers database services, BizTalk, AppFabric, WCF and workflows.
  • There will have to be new champions for his previous products. Perhaps not directly in the terms of organizational position, but in the terms of being the public face for said products. Scott Hanselman and others will probably come to the rescue web-wise.
  • Somebody else will probably keynote MIX12. Then again, PDC11 will probably see even a more empowered ScottGu preaching the Azure religion.

One thing I’m a bit worried about is the openness development that has been set well into pace in Guthrie’s previous organization. Co-operation with the jQuery team, integration of new open source technologies into the Visual Studio / .NET mix, growing support for releasing previously closed source, very open and straightforward relation to the developer scene… All this has been highly valuable.

I’m sure the Gu will take much of the goodness to his new org, but I’m less certain if the goodness will remain in the old one. At the same time, all this makes sense: ASP.NET and Silverlight are very, very mature compared to the development story on Azure. It’ll be interesting to see his impact over the coming months. Also, given the somewhat inherently closed nature of the cloud, the next steps of the openness story are worth looking forward to.

May 3, 2011 · Jouni Heikniemi · No Comments
Tags: , ,  Â· Posted in: .NET, Cloud

Valio.fi deep dive #2: Code review policy

In this second episode of my postings on valio.fi, I’ll look deeper into the concept of code reviews in the project.

Pre-post note: No, I’m not following a logical order with these. Rather, I’ll try to answer the asked questions as soon as I can while still maintaining a readable approach to the project details. Front-end posts will be interleaved with back-end magic, and there will be a dose of project management sauce inbetween. If you want to hear about something, ask for it in the comments.

In a project with numerous developers – in our case a scrum team of nine – the question of spreading code knowledge and ensuring certain quality bar tends to arise. There are numerous ways to tackle this, but we picked two principal approaches: Shared physical working space and code reviews.

Peer reviews vs. architect reviews

imageThere are basically two alternative approaches to picking the reviewer. Either you have a reviewer/architect role (which may be rotating, but often tends to stick), or you can have a system of free peer reviews, i.e. anyone can review anything as long as they feel sufficiently competent about it.

Wanting to emphasize the equality of team members, we naturally chose the peer review model. However, few teams have a totally balanced review graph. Usually some people end up reviewing more code than others. This is partially a function of seniority, but also dependent on character, preferences and random factors (such as knowledge of a particular tricky subject).

Ours wasn’t balanced either. A couple of persons ended up being a part of most of the reviews. Still, we found the system to be fairly optimal: it did distribute a lot of knowledge, everyone had their code reviewed by several others and everyone reviewed somebody’s code.

Review guidelines

As with any project with more than a few people and a duration of months or more, writing down the intended methodology felt wise. Thus, we did have written review guidelines, although they were just that – common sense was allowed to rule over these idealistic methodology statements. Here is a summary of our guidelines with some examples and clarifications in parentheses:

Changes that must be reviewed

  • Changes to commonly used base classes
  • Changes to tool libraries (defined as a group of Visual Studio projects)
  • Changes which alter the architecture (add new layers of abstraction, create new dependencies) or institute new technical vocabulary (e.g. introducing the concept of “distributed cache eviction handlers”)

Changes that are recommended to be reviewed

  • Changes which touch critical, bug-sensitive code paths (O/R mapping internals, request routing, complex logic)
  • Changes which seem incorrect or illogical by the first reading of the code
  • Changes whose unit test coverage is insufficient but hard to improve (I’ll get back to this in a later post)

Rules of play

  • The author of the code makes the first call on whether to request review. Anyone else in the team may also request a piece of code to be reviewed if he feels it necessary.
  • The author may request the review from one or more people.
  • Anyone may review, but authors should usually pick the one most experienced in the domain of the change.
  • Reviewers can ask for additional reviewers if they so feel.
  • Reviews should be handled in a speedy manner. A typical review request should be completed in a working day.
  • The author must answer any comments or questions raised during the review in a speedy and prudent manner, but it is ultimately the author’s choice to either honor or ignore them. Should significant disagreement rise, the team will solve it together.

    Some notes on the guidelines

    These rules, as written above, were applied at the start of the project. During the final few sprints, we relaxed the guidelines and focused on shipping.

    No, wait, maybe that was what you expected to hear. Actually, we tightened the rules a bit for the final three sprints (six weeks) or so. Mainly, we merged the two first headings; essentially, we started to require reviews on all non-trivial changes to the code base. We also pushed this to our project management system; a sprint backlog item was not marked complete until it had passed review (for whatever definition of “passed”; we still allowed the author to decide whether review comments warranted code changes or not).

    As you can see, we relied a lot on code authors. We allowed everybody to request review from anyone, meaning that we didn’t – nor did the project management – get involved in how much time was spent in reviews. And quite some time was spent, although we didn’t log any specific numbers. Personally, I spent almost half of my time reviewing code towards the end of the project, although admittedly I was one of the most active reviewers.

    Forced vs. recommended: Before or after checkin?

    There are basically two stages when you can do a review. Many open source projects take a fairly strict path: Only a select few people can commit or authorize commits to the source control, thus essentially creating a tight net for reviews. Often, in the largest projects, nobody commits anything without somebody else looking at it first. This makes sense when nobody can know the code authors.  The other alternative is to allow free commits, but review before releases or other milestones.

    We were totally in the post-checkin camp. Everyone could commit anything, and we had no system in place to track if people actually did request reviews (until late in the game when we made it part of the completion criteria).

    image

    If you have the right culture, reviews are free

    So yeah, we had a very author-controlled approach to reviews. But we also had a very open code quality culture, and it was not uncommon to question the validity of somebody’s approach over lunch. The debates could get quite heated, and we often spent a lot of time on issues that didn’t, in the end, have enough weight to warrant such use of time.

    However, it all served a purpose. Our somewhat excessive quality discussions created a culture where writing bad code – or skipping reviews – wasn’t particularly lucrative. You never got publicly lambasted for a stupid bug in unreviewed code, but everybody felt better collectively owning code that had passed a review: a bug in well-reviewed code was a team mistake.

    The reviews definitely had a time cost. Their positive impact is very hard to measure, given that we couldn’t know what the amount of bugs would have been without the review net. We caught some in the reviews, but probably prevented ten times more by conveying knowledge and setting examples during the review process.

    A couple of weeks after shipping the site, many people kept wondering about the low amount of bugs discovered right after the launch. This sentiment, no matter how unmeasurable, is one of the key indicators of our success in terms of quality.

    A perfect success?

    Far from it. First of all, our front-end code wasn’t nearly as meticulously reviewed, for a variety of reasons (although the same attention to quality did apply, it wasn’t as collective). There were definitely scenarios where reviews could have saved us from some front-end issues.

    Then, there was the question of review culture. While we were mostly quite successful with it, we could’ve been a bit more stringent with the adoption at the early stages. Given that the team was formed from three organizations (with three different review backgrounds), we should have made everything clear from the get go. It would have been a step away from self-organization, but we would have set a meaningful default, then allowing the team to find its own path. At the very least, review completion should have been a part of the “done” criteria from a much earlier point of time in the project.

    All in all, reviews made a big difference in the project. One more thing we could’ve done better is follow-up: We should have gone through common review comments more often and intentionally disseminate information about the issues on a weekly basis. We didn’t have a process for that. In a bigger team and a more intense schedule, this would have been even more important.

    imageUp next: Tools

    That’s it for the review methodology. The next post will be about the “how” of reviews, including the tools (Crucible). At a later stage, I will return to the topic of reviews by explaining a few of the common issues we found and giving some ideas on how to tackle those issues.

    April 28, 2011 · Jouni Heikniemi · 2 Comments
    Tags: ,  Â· Posted in: Misc. programming

    Valio.fi deep dive #1: Understanding our JavaScript philosophy

    In this first part of my series of postings on the Valio.fi project, I’ll discuss some design aspects of our use of JavaScript. We outlined these principles in the TechDays presentation, but the guidelines are worth repeating in print as well.

    What drove us to this?

    imageFirst, let us reiterate the project background and goals.

    When the concept of the site was born, high-fidelity user interface and visual appeal were high on the priority list. As you can see by browsing the site, particularly recipe search and articles, a hefty amount of work has been done to make the content look good. This includes contributions from artists, copywriters and photographers.

    Once we had all this good-looking content, the challenge moved on to the viewing experience: How can we make it as enjoyable as possible for the user to browse the content? There had to be that certain feeling of abundance, and some of that comes from speed: new content has to be available so fast you think you can never consume it all.

    But it wasn’t just viewing the content, it was also about entering it. For example, the recipe editor. Few home cooks have written down exact instructions on how to prepare the dishes they have invented. Sure, many write down the ingredients, but it’s a real effort to produce a concise set of instructions, in followable order, on how to reproduce the dish.

    What kind of user interface would provide a reasonable approach to this? A set of text boxes would work, but be very clumsy in terms of reordering entered content. A textarea might be acceptable, but then dividing the instructions into clearly printable steps would involve splitting on line feeds or other semi-technical tricks, likely to mislead the user or produce unwanted results.

    Client-side scripting shouldn’t be your first option

    Both examples above added up to this: We wanted faster search-based browsing, which called for a client-side implementation. And we wanted drag-and-drop rearrangeability for recipe entry, as other options would simply have been too unusable.

    After all the debate, there was a valid case for a JS-driven user interface. It wasn’t our first idea, nor was it an obvious choice to make. But in the end we did make that choice, and again a few more times along the way. Here are some of the aspects that affected our decisions:

    • Why do we want this? (see above)
    • Would it be possible to implement a reasonably usable non-JavaScript version?
    • If yes, would it involve a reasonable amount of work compared to the user benefit?
    • What proportion of users would suffer from the lack of JavaScript support?

    Different cases, different reasons

    I’ll present a few use cases for JavaScript on the site and try to give an idea on the arguments behind the decisions.

    image

    1) Features where JavaScript support is completely optional, just providing a better UI

    For example, login to the site. The login function is available in the footer bar – if you don’t have JavaScript, the login function just acts as a link to a login page with a standard HTML form. If JS is enabled, it renders a “toast” window, allowing you to log in within the context of the page.

    This kind of use of JavaScript is trivial to defend: it harms nobody, so the only real effect is the effort of writing and maintaining it (and testing the non-JS version, which tends to be forgotten). The vast majority of the site’s JavaScript usage falls under this category; little tricks to make the site look better and work smoother.

    2) Features where a reasonable JavaScriptless implementation is not doable

    The recipe editor is a prime example here. While it would be possible to conceive a UI that would work without JavaScript, it would be a stretch to call it usable. The recipe editor requires a lot of complex data input and fairly sophisticated input guidance. For example, we know the available units of an ingredient only after the ingredient has been selected. The selection list is long enough to make <select> population  quite unwieldy (not to mention the UI of the selection itself!).

    imageWhile there definitely are quite a few users without JavaScript support, what percentage of them would be likely to use JavaScriptless user agents to add recipes? We deemed this portion of users small enough to ignore. Sounds harsh, but it is, ultimately, a business decision.

    Many projects fear these decisions, and thus paralyze themselves with JS improvements: “if we cannot build a complete, perfect graceful degradation story, we can’t use JavaScript at all”.

    3) Features where the JavaScriptless implementation is vastly simpler in terms of features

    imageSearches are the most prominent example here. We provide a decent search without JavaScript, but it’s far from the visual glory of the JS version. It is not even functionally equivalent. For example, the JS version allows multiple search criteria from the same category (for example, you could select to search for recipes with meat AND cheese), while the pure HTML version only enables searching by one option in each category.

    Such restrictions were not always technically necessary. We could have created the search interface with lots of multi-select controls or massive checkbox groups, and thus replicate the user experience. However, we often found the simpler option more usable. For example, without JavaScript, we could not show real-time numbers of matches for each filter. Thus, the user would essentially be searching blindly, perhaps ending with zero results more often than desired.

    4) Features where we just didn’t want to bother with the scriptless implementation

    image

    Finally, there are the features which could have been implemented without scripting but were not – usually simply because we didn’t have the time or considered something else more important. Perhaps the most visible example of this was the recipe scaling feature, allowing you to change the number of portions for any recipe.

    Superficially, it’s just two buttons that scale the recipe up or down. Behind the scenes, a whole lot of things happen when scaling. We use a JSON request to get the scaled recipe ingredients from the backend and then render it on the page using jQuery templating.

    We could relatively easily add an HTTP GET based approach to scaling, providing links for +/- and then using querystring arguments for conveying the portion count. That might be something we’re looking at later on.

    TANSTAAFL

    All the sweetness from JavaScript doesn’t come without it problems, though. In exchange for bypassing some of the HTML basic form controls, we hit the following issues:

    • Accessibility degradation; impactwise ranging from mild (drag and drop without a mouse?) to severe (you just cannot scale a recipe without JavaScript support)
    • More code means more bugs; to provide a JavaScript-based UI with pure HTML fallback often means coding some things almost twice.
    • Increased complexity in terms of testing; it’s very difficult to get all the happy and unhappy paths tested, both for JS and HTML.
    • Increased attack surface from a security standpoint; while JavaScript itself rarely presents a security hazard, the additional APIs must also be checked for security issues, DoS opportunities and whatnot.
    • Considerably higher skill demands for team members; few backend developers understand the requirements for a decent JavaScript API, and while learnable on the go, it is expensive.

    All in all, there are quite a few hurdles that need to be considered. Accessibility is a very relevant and real question, but it’s a business one. We wouldn’t have that problem in an ideal world, but since few projects have the resources to actually deliver fully usable pure-HTML experiences, most will practically have to find a compromise.

    Unacceptable reasons for dissing JavaScript

    While at it, I want to mention a couple of common reasons I hear for not using client-based technology.

    • “We can’t make it work the same for everybody” – probably true, but it’s also a cop out. Check with your business owner if that’s really what you need to do.
    • “We can’t make it work the same across all the browsers” – partially true, but if your understanding comes from times before jQuery et al., do refresh yourself. Getting CSS layout to work across browsers is much harder.
    • “JavaScript is not a proper programming language” – this is a colored statement coming from somebody with a limited definition of “proper”, typically involving static typing and class-based inheritance. Rest assured, the expressiveness of JavaScript will not be the limiting factor in your project.
    • “JavaScript doesn’t perform well enough” – True in a very specific sense, but an insufficient reason for complete dismissal of client-side scripting. There are scenarios where scripting performance can kill your site. Large DOM manipulations in particular can be a headache on older browsers. You need to test, test and test. But with some work and skill, you can do a lot even with IE 6 – and most uses of JavaScript will encounter no performance issues whatsoever.

    Conclusion

    I want to end the story with a concluding note: There is a lot of power in JavaScript. As usual, not everything works – or is easy to do. But if you are building a web solution where the user experience matters, you will be doing yourself a disservice if you didn’t even think about the possibilities provided by client-side programming on the web.

    For a larger project, you would do well to devise a consistent, premeditated decision making framework for evaluating your JavaScript scenarios. Each instance of JavaScript use should provide reasonable benefit to offset the technical cost and possible accessibility issues. With that design-time check in place, client-side scripting can improve your site drastically.

    April 18, 2011 · Jouni Heikniemi · 2 Comments
    Tags: ,  Â· Posted in: Web

    MIX’11 keynote summary, day 1

    It’s time for MIX again! Microsoft’s number one conference for web and phone enthusiasts kicked off in Las Vegas. The first keynote day covered web development, the second one will zero in on phone thingies. Here are the highlights from the first day’s keynotes.

    Internet Explorer

    imageDean Hachamovitch kicks off the keynote – and he’s wearing a “TEN” T-shirt hinting at IE 10.

    • Examples of real world HTML 5 applications: Foursquare Playground, SVG animation, Director’s Cut (a tool you can use to create custom Bon Jovi music videos), World’s Biggest Pacman, …
      • Lots of boasting on IE 9’s performance, but little else. Still, it’s been a while since a new high-end HTML site recommended a Firefox user to pick up IE 9 (as Foursquare Playground does).
    • IE 9 patterns: A segment of keynote that focused on celebrating IE9’s developer readiness
      • http://html5labs.interoperabilitybridges.com/ – Microsoft’s web site on emerging
      • In the future, platform previews will appear between 8-12 weeks instead of 8 right now; Microsoft feels this will provide more efficient changes and ability to hear the community feedback.
    • Looking forward: IE10 has been developed for three weeks now
      • Improved CSS3 support (visual effects, columns, grid layouts, …). High speed is still a priority, and heavily demoed.
      • Platform Preview 1 is now available for download
      • IE10 demos were run on an ARM machine with 1 GB of memory, demoing the capacity to run (fast) on an ARM machine.
    • PDC 2011 will be held on 13th-16th September in Anaheim, California (the PDC site isn’t up to date yet though).

    Developer tooling and server end

    Scott Guthrie on the stage, turning the focus back into the server end.

    • Now shipping ASP.NET MVC 3 Tools Updateimage
      • New project templates, including a “use HTML5” switch, NuGet packages pre-installed
      • Entity Framework 4.1 (including code-first capabilities)
      • New jQuery version 1.5 shipping
      • Modernizr – another open source library shipping with Visual Studio through NuGet. This one makes it easier to use HTML5 and still have a working down-level experience.
      • New scaffolding support: Creating a code-first model also creates controllers, views and whatnot.
      • Demo: Creating a simply administrative interface to a podcast. The highlight of the demo was definitely building a code-first model, which then resulted in a generated database, controllers and views, including database constraints and values. Not useful everywhere, but certainly helps in quite a few scenarios.
    • WebMatrix demo: Building a front-end. Demo included buying a web site template from TemplateMonster and using it to whip up a quick web site with WebMatrix, including lots of easy helpers for social features, packge-based helper installation and other stuff.
    • Orchard (an open-source CMS to which Microsoft is contributing code): image
      • Again, returning to WebMatrix, and building a site using Orchard downloaded from the Web Gallery.
      • Version 1.1 now available; the new version is definitely much cleaner than the previous ones.
      • Also heavy demoing of Orchard modules, downloadable and installable pretty much like WordPress plugins. Potential new business opportunity, once all the Microsoft-savvy developers get into WordPress-style development?
    • Windows Azure: Releasing new versions of Access Control Service, Caching and Content Delivery Network, and a feature called Traffic Manager. Not much detail in the keynote though, but look forward to the coming days.
    • Umbraco project founder on stage to discuss the CMS:
      • 10 000 active developers around the world – not bad for an ASP.NET open source project
      • Touting Windows Azure support: Umbraco supports automatic scaling, allowing the admin to specify tolerances. Umbraco will then adjust the number of Azure nodes needed to serve the site properly. Now downloadable: Windows Azure Accelerator for Umbraco

    April 12, 2011 · Jouni Heikniemi · One Comment
    Tags: , , , , , ,  Â· Posted in: .NET, Web

    Out of the dark

    It’s been a silent couple of months. Offbeat Solutions has been heads down programming, debugging, optimizing – and then, finally shipping. Now it’s time to look back.

    imageIn TechDays Finland 2011 we finally talked about our latest customer project, the www.valio.fi web site (slides available, in Finnish). In case you’re not from around here, Valio is the Finland’s leading company in dairy products and a consumer brand of legendary caliber.

    Since the presentation on 31st March, we then finally made the site live on Saturday 2nd April. If you can read Finnish and like food, go check the site out at www.valio.fi!

    What’s inside the box?

    Essentially we – including our partners in crime, Valve Branding and Appelsiini Finland – delivered an ASP.NET MVC 3 application with quite a few business system integrations and a SQL Server database. Woohoo – that’s not really the exciting part. But we truly did deliver something that’s not the usual user (or developer) experience, so what’s different?

    In my opinion, there are six aspects that differentiate the Valio site and the implementation project from most cases developers get sucked into.

    1. User experience. We use modern browser capabilities to make the user experience smoother, faster and visually more appealing. And we do mean so much more than slapping an UpdatePanel here and there.
    2. Platformness. Our task was to implement not just a web site, but also a platform for many future digital marketing endeavors. For this purpose, we expose a whole lot of APIs – and use them ourself.
    3. Automated testing. Although we are nowhere near full test coverage, our test suite compares favorably with most web projects.
    4. Manageabilility. We provide decent tooling to manage the site, reducing the need to type custom SQL queries. Yeah, we have PowerShell commandlets.
    5. Scale. Valio.fi isn’t huge, but it’s running on a cluster of web nodes. It has all the challenges of being on a cluster, most of which never occur on a single-node web site.
    6. Project methodology. We had imperfect Scrum and all, but we had successful teamwork with almost 20 people involved daily – and with relatively few communication issues.

    Coming up: More info

    Thanks to the openness of Valio and other involved parties, we enjoy pretty broad liberty to discuss the technical specifics of the project. Offbeat Solutions wishes to use this opportunity to disseminate information and best practices on building modern web sites with ASP.net.

    During the coming months, we intend to blog on many of the subjects mentioned above. If you have feedback or want to know something about the project, please do leave a comment!

    April 8, 2011 · Jouni Heikniemi · 6 Comments
    Tags: ,  Â· Posted in: .NET, Web