HN tl;dr: innovation in the 2010s means getting your anonymized social network for marmots acquired so that big SV can put ads on it or datamine your users
proggit tl;dr: you should have spent your time learning how to write a CRUD app and a mediocre template system, dependency injection framework, and ORM in Haskell Rails Erlang Scala Rails Python Node Clojure Julia instead of mastering one tool and learning how to solve problems.
It Begins
I shut down my music career in 1998. "Career" is too generous a word. I'd
made money as a musician, but not enough to call myself a professional and
certainly not enough to pay the rent and buy food. I talked my way
into the Color LaserJet group at HP.
I wrote some code for the printer group, just because I could. I had a
little AWT app (even though Swing had just come out, it wouldn't run on Linux
for quite a while) for customer support techs to record data about customer
problems. Then the networking group asked me to write a program to email
customers whenever the web site had a new technical article published. The ten
line shell script (written in whatever HP-UX 9.x supported out of the box) was
orders of magnitude less than the Java version I could never convince myself to
finish.
When I realized that I enjoyed playing with HP-UX and the Linux box under my
desk far more than I appreciated learning all of the ways that paper could jam
or PostScript drivers could render things wrong, I switched to system
administration. (Programming was in a different state altogether, and my music
degree hardly compared to the CS degree they wanted.)
After fighting fires for three months and automating away the remaining
problems, I had a lot of time on my hands. The web was suddenly very
interesting. mod_perl had just come out and applets were obviously not working.
I picked up Perl.
I started a couple of small free software projects on my own. They never
went anywhere. (Everyone used to write a template system in those days. Then I
found the source code to Everything 2's
predecessor, sent in a couple of patches, and (after Carly Fiorina decided to
save money by not paying system administrators) eventually started working for
the dot com that produced Perl Monks and
promptly imploded.
Then I wrote a book. It had a cool cover but you probably didn't read it.
(It was only the first book in English about blogging, but the title was wrong
anyhow. This was the last gasp of the first dot com era, when everything seemed
like a good idea in a certain context, almost including the TSA.)
Writing a book is equal parts frantic research and tedium, and in the first
of many moves designed to convince myself that my actions had a positive effect
on a cruel and uncaring universe, I took up Michael Schwern's call to unify the
internals of Test::Simple and Test::More. The result is
Test::Builder.
This was a revolution for everyone on the perl-qa list and it was a
revolution for Perl in general, but it's the kind of revolution that you look
back on now and think "Wow, that was obvious!" and "That was a revolution?"
because it seems like such widespread common sense, like "It's good to drink
water!" and "Maybe smoking has deleterious health effects."
It was a revolution in that it let a thousand blossoms of test modules bloom
(so people could solve their own specific testing problems and share those
solutions) while providing the soil in which all of those blossoms could bloom
(so they all worked together with almost no effort on the module writer's
part). This is an important principle of API design: handle the difficult,
tedious, and tricky parts at the low levels so that everyone gets that right
while not ruling out people doing their own things at higher levels. If that
makes you think of layering, you're right.
I did some consulting work for a while, based on the strength of my free
software work with Perl. (Making a name for yourself answering questions on a
site does help get your name in the community and build some networking
connections. Getting a patch or two in a dominant project helps as well.)
Then my career took a lurch sideways, and I spent some time as an editor
concentrating on free software and technology.
Free Software as a Hobby
It was important to keep my programming skills fresh, so I increased the
amount of time spent contributing to free software projects (in and around writing books). Parrot sucked up a lot of my efforts.
(More than one Parrot/Rakudo/P6 developer said to me in separate
conversations as far back as 2007 that "Sure, it's a long slog now and no one's
getting compensated much for it, but when it's done in the next 18 months,
there'll be a lot of interest in books/training/consulting, and we'll be in a
good position then!")
This was a rough time to be in Perl. Having helped push the Ruby snowball
back down the hill with Rolling with Ruby on
Rails (just over nine years ago now!) and with Perl stuck on 5.8 for
another almost three years, the hope was still that a new major version would
turn the language's fortunes around. Meanwhile, the testing culture had
infected the core language as well as the CPAN like an unseen brigade of
healthy gut fauna (don't leave the dinner table without it). There was growth
and roots and foundations, but the effects would take some time.
Pugs helped, but Pugs has been a historical curiosity for a long time
now.
The Resurgence of Perl that Nobody Noticed
While the world stopped waiting for a new major version of Perl, Perl 5 came
out with several new major versions and the CPAN continued to be the reason
that I stuck with Perl. I'm not alone in this. Moose is probably the singular reason there
are any significant new projects in Perl in 2014. This by no means should take
credit away from other projects such as DBIx::Class, Mojolicious, Catalyst, and other projects well
worth mentioning, but Moose was a revolution in the noisy, flashy, overthrow
the dominant paradigm sense. It was Fidel to the rest of the CPAN's
Che—the one that gets the credit, unless you're some sort of weird
countercultural rebel who likes giving oligopolic corporations money to wear
t-shirts with your icon's image on them. (That metaphor rode away from my point
on a motorcycle tour of South America. Sorry.)
That revolution needed a name.
Modern Perl
JavaScript
had its resurgence, going from "that awful language that everyone has to
use and nobody likes" to "Sure, if you squint hard enough and ignore all of
this awfulness over here and are very very disciplined, you can write code
that's not too bad". (If that apologia sounds familiar and you feel bitter,
keep in mind that, unlike Perl, you have no choice whether to use
JavaScript on the client side.)
I figured this was a great time to leave the increasingly depressing world
of corporate technical book publishing to work on my own technical book publisher, because it
didn't cost us tens of thousands of dollars to get the first copy out the door.
While it's nice that the revenue we make on a book like Modern
Perl, it's still a lot better to think of writing or publishing a book as
"a hobby that'll buy you nice dinners here and there" instead of "I will never
have to work again, unless you count sleeping on a pile of gold coins to be
work".
Book publishing is driven by the semi-random lightning strikes of big hits,
and you have to be in the right place at the right time. Like a poker player,
you need to know when to go all in—and, more important, you need to know
when to get out. This is one of the reasons the tech book publishing market is
so awful. Ruby and Rails were the hottest things since JSP in 2005 and 2006,
but the market had reached its peak of sales in 2007 even as publishers were
still pumping out the kind of ancillary books around Rails that they'd pumped
out around Perl in 1998, 1999, 2000, and 2001 even as talking puppets,
razorfish, and drones delivering fresh vegetables even before you one-click
order them had become a bubble about to pop, taking with it countless jobs and
lots of paper money in an inflated stock market. (Maybe that's what you should
expect from hiring a member of an adolescent power fantasy fiction cult to
manage the world's largest economy, but keep in mind music degree not economics
degree and everything's more obvious in retrospect.)
That's one reason why there hasn't been a Modern Perl Testing book;
I can't convince myself to give up increasingly valuable weekends and evenings
when there are other things I could do which are either much more relaxing
(spending time with friends and family), much healthier (cooking, exercising),
or profitable (investing). At some point, there might be a
Programming Rust book, if only because I find Rust fascinating for what it both does and
doesn't do. I wish I'd had it ten years ago for Parrot.
With that said, writing and publishing Modern Perl was a good
thing. Giving it away for free was a good thing. Maybe I would have made as
much revenue if I'd spent the same amount of time slinging milkshakes at the
local SBUX, but I'm more satisfied about the social good done by giving away
PDFs and hosting a web site than I am about most anything else done in my
career. (Perhaps Test::Builder has been
more influential from a code standpoint, but once I start going down that line
of thought, I feel a complex
insert-lengthy-German-compound-philosophical-term-here emotion about those
large projects I worked on which seemed influential and important at the time
and haven't actually gone anywhere.)
At worst, Modern Perl gave Perl as practiced by its best
programmers in the 2010s a concrete name. At best, it helped existing Perl
programmers and hopefully some new ones avoid some of the worst parts and adopt
some of the best parts.
Golden Handcuffs For a Very Modern Perl Consultant
With a brief foray into running my own non-publishing businesses (hey, if
you want to give this free
investing articles for novices a link somewhere, that'd be great) revealing
that starting a business from nothing is very, very difficult.
This, I think, is one reason why so many small consultancies are
consultancies and never manage to turn their brilliant ideas into shipping
products because it's too lucrative to chase consulting dollars and too
difficult to invest in products which may not pay off. This isn't the first
time I ran into this, either. If my first job after the dot com crash had
succeeded in producing its product, it would have owned a lucrative vertical
market. Unfortunately, consulting dollars ran out before it could finish a
proof of concept to jumpstart presales. (Of course, the investors also wanted
to rewrite the Perl backend and the Python/Wx client application in Java, in
2002, to run on point of sale systems for small businesses, in 2002.)
When consulting is good, it can be lucrative and that lucre can be tempting.
Consultants have to charge a rate far in excess of a normal hourly salary
because there's overhead: you may have a lot of spare time between contracts,
you don't get PTO or benefits you don't pay for yourself, and you have to
negotiate contracts and pursue work which may or may not materialize.
If you find a good client, you treat him or her like a combination of the
Pope, the Queen, and the Dalai Lama, because a trustworthy client who pays you
on time and doesn't argue over little details of the contract is better than
gold.
Yet suppose you budget for eight or nine months of steady work and luck into
a contract that gives you twice that. Some people deal well with that boom and
bust. Others don't. (For me, the mechanics of budgeting are easy but the
psychology of riding the business cycle is difficult.) It's difficult to look
at other opportunities with an objective eye because when the contract is
great, that money looks very nice and it represents a lot of freedom, but when
the contracts are awful or far between, the stability of a salaried job
represents a lot of security.
Finding a Perl Job
Am I a Perl programmer?
If you look at my résumé, I've been fortunate (in one sense) to have spent
the past 16 years working with Perl at a deep level on many, many projects in
many jobs in many domains. I've also made money working with Python, SQL,
shell, Ruby, C, C++, JavaScript, assembly, Java, SQL, PL/pgSQL, Visual Basic,
PHP, LaTeX, and I'm sure I left out at least one or two.
As a professional programmer with an interest in my professional development
and someone who is physiologically unable not to program occasionally
outside of 9-5, I've dabbled in other technologies—not enough to call
myself an expert the way I call myself a Perl expert—but enough that I
don't completely embarrass myself in conversation about them. (Chuck Moore once
tried to convince me of the value of writing my own operating system. I have an
MP3 of this.)
Yet on my CV, one technology keeps coming up again and again.
Perl
It's no secret that that's where I've spent most of my time. Look, I
like the language. I'm used to its flaws (even though a couple of
hundred entries on this site will reveal that I don't like all of them) and I
know how to play to its strengths. I like its community (people like Tim Bunce
and Rik Signes and Andreas Koenig and Tatsuhiko Miyagawa are, frankly,
irreplaceable elsewhere). I'm exceedingly productive with Perl in a way that
would take me a while to get productive in other languages. That's what you get
from a decade and a half of experience. That's what you get from at least
20,000 hours of practice.
But Did You Learn to Program?
You tell me. I can give you references who will speak of me in positive
terms. I can refer to to people who will tell you I helped solve problems or
helped teach them something or helped them think about their problems in better
ways.
(You'll also find people who will say I can be sarcastic and moody and
bitter, and for better or worse you're reading this, so why pretend?)
This is the important question that a CV can't answer. This is an
important question that rifling through the small subset of my professional
work that's on Github or the CPAN won't tell you. They won't tell you about all
of the mistakes I made that you don't see, if they exist. They won't tell you
about all of the thrashing around that I did or didn't do which Git let me
squash out of existence. They won't tell you if I churned out 1400 lines of
code one afternoon in a marathon coding session and it compiled on my second
try and hasn't been touched since, because it just works and no one
needs to touch it.
Then again, asking me to write a sorting algorithm on a whiteboard won't
tell you much either beyond "Does this person actually seem like he knows how
to program at all?" and that's still a thing in
interviews.
You can't look at a CV and tell whether I'm the kind of person who will go
above and beyond the task of transcribing what a software architect dictates
that the project manager has teased out of what the director of sales has
forced the enterprise sales representative to admit that he promised to the
customer for the next release. (I can tell you that I would find that
environment frustrating.) You can't predict only from a page and a half whether
I have an enthusiasm for writing the best darn medical insurance billing code
ever or whether that enthusiasm will translate into an attempt to delight users
even if they never think about all of the bugs and frustrations that we
squashed and eliminated before they escaped to production.
You can't tell that by reading Perl or Ruby or
PostgreSQL on my résumé. You can't even tell that by reading my code.
That's the risk you take in hiring someone, but isn't that at least as
important as answering the question "Can you make a decent attempt to write
something resembling code on a whiteboard, even though your muscle memory has
been tuned over several years for Vim, because you're ostensibly a
professional who cares about tools and automation and making it easy
to do the easy work of typing?" or the question "How well do you fit with our
team?"
But Where are the Perl Jobs?
I'm not a unique and special snowflake. Programmers aren't
fungible, but there aren't so many difficult programming tasks that you need
the world's singular expert in a subject. (The fact that my degree says Music
and not "Computer Science" from Stanford, Berkeley, MIT, or CMU probably keeps
me out of computer vision, AI research, and compiler design, however, just as
much as it's garlic to the Google recruiters who want me to believe it's an
honor to spend weeks answering questions about ping pong balls and sorting
phone books for the opportunity to babysit a data center full of machines
dedicated to slapping ads on the world's information.)
It should be no secret that, if I stay in programming, my preference is to
work with the language of my expertise and preference. I believe that the
fashion-driven chasing of programming fads is bad for business and programmer
alike, but that world isn't changing any time soon.
As I see it, you can find a Perl job in one of a few ways:
Create it yourself by starting your own company
Create it yourself by recruiting a client without a strong technology preference
Create it yourself by introducing Perl slowly into an existing job and making it irreplaceable
Find one of the existing companies using Perl (read "Move to Amsterdam" or "spend time maintaining a codebase from 1997" or "get very lucky")
Create something so amazing and compelling and useful that people can't not use it, like mod_perl or Movable Type or, let's throw cPanel and Catalyst in there
...
Fifteen or fourteen or even ten years ago, I might have railed against the
inherent unfairness of a cruel and uncaring universe of technology driven by
fashion, where technical superiority is less important than buzz, but I've
tried these approaches over my career. As I ponder the midpoint of my career
(asking myself "Should I get a CS degree?" or "Wouldn't it be more effective to
get a degree in finance and manage a boutique legal firm?"—I am not
making this up, but then again I thought about buying a bakery a couple of
years ago and it took Robert Buels
only a few minutes to talk me out of it), the broader question of patterns and
trends comes up.
Is Any Technology Immune to This?
Apart from COBOL? Mumps? No.
Do I want to program in Cobol or Mumps for the rest of my life, or even the
rest of the afternoon? No.
Look, I'm a programmer. I solve problems. I've managed teams and I've been
managed. I've written documentation and I've written tests. Some days I've
deleted more code than I've written and I've been glad of it. Some days I've
solved customer problems by not writing code. Some days I've been
elbow deep in profiles or Valgrind reports and other days the best thing I've
done was add an index to a column in a database and it was totally worth all of
the research to reach the point where that was the solution.
I have my preferences in technology, sure. I also have my experiences. When
you hire a programmer, you're getting his or her preferences and
experiences and biases.
Maybe I'm being too picky. Maybe it's natural to reach the mid-career crisis
and say "Arguing over which syntax is best on the Internet is a poor use of
time I could spend walking a dog or playing pinball with my eleven year old
nephew or celebrating a housewarming with friends". Maybe it's acceptable to by
cynical and say "I'm a professional, which means that even if I have no
personal interest in managing clinical research records, you are
paying me enough to bring my knowledge and experience and hard-won judgment to
bear to solve this problem in an effective and trustworthy way", but I'm
probably not going to spend my nights and weekends thinking about it.
Alternately, perhaps I should have bought that bakery. Sure, you
have to hire people you can trust and pay them a lot of your revenue and you
have to get up at 4 am every day and health inspectors and land zoning and the
prices of dairy products...
... but you're not dealing with VCs who underpay you and dangle ever more
diluted RSUs in front of you for that 1% chance of an acquihire payout that'll
slightly pay more than if they'd paid you a market rate for the start, or
clients who want the world in a week for $20 an hour because some teenager in
Albania only charges $8 an hour but at least you speak English, or an HR
department that sees "Perl" sprinkled liberally through your résumé and thinks
either you're the system administrator you started as in 1998 or someone who
fell through a time warp in 1999, and by the way, they have the Internet on
computers now, har har, JavaScript is a real language, would you kindly join us
in the new spring lineup for 2014?
The Real Question
For someone whose life obligations preclude selling everything and living on
a beach in Thailand or Belize for a year, how do you evaluate a career
reaching its midpoint? How do you keep programming fresh? How do you market
yourself as someone who solves problems instead of someone who
transcribes ideas in the language du jour? Or do you leave programming
altogether?
I don't have the answers, but if there's anything a decade and a half of
experience has given me, it's the ability even to ask those
questions.