One thing programmers are notorious for is their hatred of deadlines. They don’t like making estimates either, and that makes sense, because so many companies use the demand for an estimate as a “keep ‘em on their toes” managerial microaggression rather than out of any real scheduling need. “It’ll be done when it’s done” is the programmer’s gruff, self-protecting, and honest reply when asked to estimate a project. Whether this is an extreme of integrity (those estimates are bullshit, we all know it) or a lack of professionalism is hotly debated by some. I know what the right answer is, and it’s the first, most of the time.
Contrary to stereotype, good programmers will work to deadlines (by which, I mean, work extremely hard to complete that project by a certain time) under specific circumstances. First, those deadlines need to exist in some external sense, i.e. a rocket launch whose date has been set in advance and can’t be changed. They can’t be arbitrary milestones set for emotional rather than practical reasons. Second, programmers need to be compensated for the pain and risk. Most deadlines, in business, are completely arbitrary and have more to do with power relationships and anxiety than anything meaningful. Will a good programmer accept business “deadline culture”, and the attendant risks, at a typical programmer’s salary? No, not for long. Even with good programmers and competent project management, there’s always a risk of a deadline miss, and the great programmers tend to be, at least, half-decent negotiators (without negotiation skills, you don’t get good projects and don’t improve). My point is only that it can happen for a good programmer to take on personal deadline responsibility (either in financial or career terms) and not find it unreasonable. That usually involves a consulting rate starting around $250 per hour, though.
Worth noting is that there are two types of deadlines in software: there are “this is important” deadlines (henceforth, “Type I”) and “this is not important” deadlines (“Type II”). Paradoxically, deadlines are assessed to the most urgent (mission critical) projects but also to the least important ones (to limit use of resources) while the projects of middling criticality tend to have relatively open timeframes.
A Type I deadline is one with substantial penalties to the client or the business if the deadline’s missed. Lawyers see a lot of this type of deadline; they tend to come from judges. You can’t miss those. They’re rarer in software, especially because software is no longer sold on disks that come in boxes, and because good software engineers prefer to avoid the structural risk inherent in setting hard deadlines, preferring continual releases. But genuine deadlines exist in some cases, such as in devices that will be sent into space. In those scenarios, however, because the deadlines are so critical, you need professional project-management muscle. When you have that kind of a hard deadline, you can’t trust 24-year-old Ivy grads turned “product managers” or “scrotum masters” to pull it out of a hat. It’s also very expensive. You’ll need, at least, the partial time of some highly competent consultants who will charge upwards of $400 per hour. Remember, we’re not talking about “Eric would like to see a demo by the 6th”; we’re talking about “our CEO loses his job or ends up in jail or the company has to delay a satellite launch by a month if this isn’t done”. True urgency. This is something best avoided in software (because even if everything is done right, deadlines are still a point of risk) but sometimes unavoidable and, yes, competent software engineers will work under such high-pressure conditions. They’re typically consultants starting around $4,000 per day, but they exist. So I can’t say something so simple as “good programmers won’t work to deadlines”, even if it applies to 99 percent of commercial software. They absolutely will– if you pay them 5 to 10 times the normal salary, and can guarantee them the resources they need to do the job. That’s another important note: don’t set a deadline unless you’re going to give the people expected to meet it the power, support, and resources to achieve it if at all possible. Deadlines should be extremely rare in software so that when true, hard deadlines exist for external resources, they’re respected.
Most software “deadlines” are Type II. “QZX needs to be done by Friday” doesn’t mean there’s a real, external deadline. It means, usually, that QZX is not important enough to justify more than a week of a programmer’s time. It’s not an actual deadline but a resource limit. That’s different. Some people enjoy the stress of a legitimate deadline, but no one enjoys an artificial deadline, which exists more to reinforce power relationships and squeeze free extra work out of people than to meet any pressing business need. More savvy people use the latter kind as an excuse to slack: QZX clearly isn’t important enough for them to care if it’s done right, because they won’t budget the time, so slacking will probably be tolerated as long as it’s not egregious. If QZX is so low a concern that the programmer’s only allowed to spend a week of his time on it, then why do it at all? Managers of all stripes seem to think that denying resources and time to a project will encourage those tasked with it to “prove themselves” against adversity (“let’s prove those jerks in management wrong… by working weekends, exceeding expectations and making them a bunch of money”) and work hard to overcome the gap in support and resources between what is given and what is needed. That never happens; not with anyone good, at least. (Clueless 22-year-olds will do it; I did, when I was one. The quality of the code is… suboptimal.) The signal sent by a lack of support and time to do QZX right is: QZX really doesn’t matter. Projects that are genuinely worth doing don’t have artificial deadlines thrown on them. They only have deadlines if there are real, external deadlines imposed by the outside world, and those are usually objectively legible. They aren’t deadlines that come from managerial opinion “somewhere” but real-world events. It’s marginal, crappy pet projects that no one has faith in that have to be delivered quickly in order to stay alive. For those, it’s best to not deliver them and save energy for things that matter– as far as one can get away with it. Why work hard on something the business doesn’t really care about? What is proved, in doing so?
Those artificial deadlines may be necessary for the laziest half of the programming workforce. I’m not really sure. Such people, after all, are only well-suited to unimportant projects, and perhaps they need prodding in order to get anything done. (I’d argue that it’s better not to hire them in the first place, but managers’ CVs and high acquisition prices demand headcount, so it looks like we’re stuck with those.) You’d be able to get a good programmer to work to such deadlines around $400 per hour– but because the project’s not important, management will (rightly) never pay that amount for it. But the good salaried programmers (who are bargains, by the way, if properly assigned or, better yet, allowed to choose their projects) are likely to leave. No one wants to sacrifice weekends and nights on something that isn’t important enough for management to budget a legitimate amount of calendar time.
Am I so bold as to suggest that most work people do, even if it seems urgent, isn’t at all important? Yes, I am. I think a big part of it is that headcount is such an important signaling mechanism in the mainstream business world. Managers want more reports because it makes their CVs look better. “I had a team of 3 star engineers and we were critical to the business” is a subjective and unverifiable claim. “I had 35 direct reports” is objective and, therefore, more valued. When people get into upper-middle management, they start using the terms like “my organization” to describe the little empires they’ve built inside their employers. Companies also like to bulk up in size and I think the reason is signaling. No one can really tell, from the outside, whether a company has hired good people. Hiring lots of people is an objective and aggressive bet on the future, however. The end result is that there are a lot of people hired without real work to do, put on the bench and given projects that are mostly evaluative: make-work that isn’t especially useful, but allows management to see how the workers handle artificial pressure. Savvy programmers hate being assigned unimportant/evaluative projects, because they have all the personal downsides that important ones do (potential for embarrassment, loss of job) but no career upside. Succeeding on such a project gets one nothing more than a grade of “pass“. For a contrast, genuinely important projects can carry some of the same downside penalties if they are failed, obviously, but they also come with legitimate upside for the people doing them: promotions, bonuses, and improved prospects in future employment. The difference between an ‘A’ and a ‘B’ performance on an important project (as opposed to evaluative make-work) actually matters. That distinction is really important to the best people, who equate mediocrity with failure and strive for excellence alone.
All that said, while companies generate loads of unimportant work, for sociological reasons it’s usually very difficult for management to figure out which projects are waste. The people who know that have incentives to hide it. But the executives can’t let those unimportant projects take forever. They have to rein them in and impose deadlines with more scrutiny than is given important work. If it an important project over-runs its timeframe by 50 percent, it’s still going to deliver something massively useful, so that’s tolerated. What tends to happen is that the important projects are, at least over time, given the resources (and extensions) they need to succeed. Unimportant projects have artificial deadlines imposed to prevent waste of time. That being the case, why do them at all? Obviously, no organization intentionally sets out to generate unimportant projects. The problem, I think, is that when management loses faith in a project, resources and time budget are either reduced, or just not expanded even when necessary. That would be fine, if workers had the same mobility and could also vote with their feet. The unimportant work would just dissipate. It’s political forces that hold the loser project together. The people staffed on it can’t move without risking an angry (and, often, prone to retaliation) manager, and the manager of it isn’t likely to shut it down because he wants to keep headcount, even if nothing is getting done. The result is a project that isn’t important enough to confer the status that would allow the people doing it to say, “It’ll be done when it’s done”. The unimportant project is in a perennial race against loss of faith in it from the business, and it truly doesn’t matter to the company as a whole whether it’s delivered or not, but there’s serious personal embarrassment if it isn’t made.
It’s probably obvious that I’m not anti-deadline. The real world has ‘em. They’re best avoided, as points of risk, but they can’t be removed from all kinds of work. As I get older, I’m increasingly anti-”one size fits all”. This, by the way, is why I hate “Agile” so vehemently. It’s all about estimates and deadlines, simply couched in nicer terms (“story points”, “commitments”) and using the psychological manipulation of mandatory consensus. Ultimately, the Scrum process is a well-oiled machine for generating short-term deadlines on atomized microprojects. It also allows management to ask detailed questions about the work, reinforcing the sense of low social status that “conventional wisdom” says will keep the workers on their toes and most willing to work hard, but that actually has the opposite effect: depression and disengagement. (Open-back visibility in one’s office arrangement, which likewise projects low status, is utilized to the same effect and, empirically, it does not work.) It might be great for squeezing some extra productivity out of the bottom-half– how to handle them is just not my expertise– but it demotivates and drains the top half. If you’re on a SCRUM team, you’re probably not doing anything important. Important work is given to trusted individuals, not to SCRUM teams.
Is time estimation on programming projects difficult and error-prone? Yes. Do programming projects have more overtime risk than other endeavors? I don’t have the expertise to answer that, but my guess would be: probably. Of course, no one wants to take on deadline risk personally, which is why savvy programmers (and almost all great programmers are decent negotiators, as discussed) demand compensation and scope assurance. (Seasoned programmers only take personal deadline risk with scope clearly defined and fixed.) However, the major reason for the programmers’ hatred of deadlines and estimates isn’t the complexity and difficulty-of-prediction in this field (although that’s a real issue) but the fact that artificial deadlines are an extremely strong signal of a low-status, unimportant, career-wasting project. Anyone good runs the other way. And that’s why SCRUM shops can’t have nice things.
