This is a series of posts about how software development and programmers have become increasingly reliant on developer communities like Stack Overflow and search engines to help them develop and ship code. We think this is a fundamental shift in the nature of software development that has wide ranging implications.
- Part 1 - A Story
- Part 2 - Diagnosing SAP
- Part 3 - How Many Programmers are SAPs
- Part 4 - Causes & Effects of SAP
- Part 5 - Refactoring the Programmer
Part 2 – Diagnosing SAP
A little while after coming to terms with my Search Addicted Programmer (SAP) condition, I got the opportunity to come work on really cool software engineering problems with the Chisel group. I decided it was probably a good time to come clean about my SAP problem and to do it with numbers by studying myself and some my fellow SAPs. So I conducted a small informal study looking at 3 developer’s browser histories over 20 days (60 developer days) to quantify how often we used search engines for programming related queries.
(Note: We were originally intending to make this into a full blown study but that project fell through. If anybody is interested in following up on this research let me know.)
| Total Searches | Programming Related | Percent | |
|---|---|---|---|
| Developer #1 | 423 | 143 | 33.81% |
| Developer #2 | 524 | 145 | 27.67% |
| Developer #3 | 799 | 710 | 88.86% |
| Total | 1746 | 998 | 57.16% |
To keep things simple and to protect the privacy of the developers, we only looked at history entries that referenced a search engine, and each developer was allowed to review and edit their browser history beforehand.
Average relevant queries per team per day
Average relevant queries per developer per day
What we found showed just how prevalent the SAP problem had become on my team. Almost 1000 total programming related searches, and several hundred searches per developer in only 20 days. Averaging this out we see that individual developers were executing 16 programming related searches per day (at least 2 an hour) while the team was executing almost 50.
To try and quantify what this meant in terms of developer time, I used a (finger in the air, we can argue in the comments) estimate of 5 minutes per query to approximate the number of developer hours per day spent searching. This is where things start to get interesting. Based on this small study of a small number of developers over a relatively short period of time, we get an estimate of over 1.2 hours per day per developer spent searching on the web, or over 4 hours for the 3 developers per day.
Individual developer per day (5 minutes per query)
Development team per day (5 minutes per query)
Every time I look at these figures I am shocked and think 5 minutes per query is surely too much, but then I think about all the times that I have spent much longer than 5 minutes debugging some nasty problem with search and Stack Overflow and think it could be much worse. Again I have no empirical data for the 5 minutes finger in the air estimate (if you have a paper or are interested on working on it let me know), but if anything it feels a little on the conservative side.
While we can debate the impact of search in terms of developer hours per day, the raw number of queries executed marks web search (and the developer communities like Stack Overflow to which it enables access) as a very important part of a software developer’s toolkit. In fact we can probably pretty safely state that developer communities like Stack Overflow, indexed and made available through powerful search engines, have become anticipated/required parts of the software development landscape, depended on not just by programmers but by software development companies and vendors alike. In the next post we try to estimate the numbers of programmers that are SAPs. (I'll stop making that joke soon, I promise.)
(For a more in depth study of programmer browser histories, Chris Parnin has since done a much more comprehensive study of the role of web resources in the practices of android developers).