Saturday, September 05, 2015

I think I need some career advice

I left Twitter several weeks ago. It looks like I need to not only decide where I want to work next, I might also need to decide what I want to specialize in as a programmer. Apparently, there's less and less demand for full-stack web developers or generalists. Most open positions are for specialists.

I've always been a polyglot. I really like learning new languages. Even though I'm mostly a Python expert, I love to learn as many new languages as possible. For instance, I accomplished the following just while I was at Twitter:

  • I built a custom supply chain management system in Python.
  • I technical edited two Scala books.
  • I learned Go fairly well.
  • I learned Android at a beginner to intermediate level.
  • I learned Objective-C and iOS at a beginner to intermediate level.

Another thing that sets me apart from most programmers is that I'm a friendly extrovert, and I like giving talks. Given the above, developer relations is a natural fit for me. I did that for two years at Google.

However, it seems like that's not enough these days. It seems like most companies and most recruiters are really only looking for specialists. It seems like these are the specialties I could conceivably go after:

  • Ruby backend programmer
  • Go backend programmer
  • Scala backend programmer
  • Developer relations (mentioned above)
  • Frontend developer (JavaScript, Angular, React, etc.)
  • Distributed systems engineer (cloud computing, AWS specialist, etc.)
  • SRE
  • DevOps engineer
  • Data scientist (machine learning, big data, etc.)
  • Hadoop specialist
  • Android engineer
  • iOS engineer
  • Software engineer in biotech
  • Contractor or consultant

As I mentioned above, I know lots of languages, including Python, Ruby, Go, and Scala. I know JavaScript and some frontend development (at least using the old-school approaches). I know Java and some Android programming. I know Objective-C and some iOS programming. However, aside from Python, I am not an expert in any of these things.

Developer relations is fun, but I'm not sure it's a good idea to bank my entire future on it.

Distributed systems, cloud computing, AWS, SRE work, DevOps, etc. are all interesting to me. However, I'm not sure I can convince anyone to give me a job doing these things. Furthermore, I'm also not sure these things are the right thing to bet my career on. On the other hand, I'm a lot more interested in cloud computing than, say, frontend work.

I have an interview at a biotech company. That could be promising. That could be really good for my career, or a dead end for my career. I'm not sure.

I'm reluctant to become a contractor or consultant. It seems to me that once you become a consultant as an older programmer, it becomes very hard to get hired as a normal engineer at many companies. One of my buddies is a brilliant older programmer (UNIX®, MIT AI lab, various wireless hardware and software systems, etc.) but companies like Google won't hire him anymore. I think being a contractor or consultant has a lot of pros and a lot of cons. I can't even wrap my head around whether or not it's a good idea to start doing.

I suspect one benefit of being a specialist is that it makes interviewing easier. If you only have to know a particular field really well, it's not that hard. In contrast, I read Python Weekly, Ruby Weekly, Go Weekly, Scala Weekly, JavaScript Weekly, HTML5 Weekly, etc. Trying to keep up with so many areas means that I can't delve too deeply into any of them aside from Python.

Furthermore, I've noticed that the fewer people have a skill, the easier it is to get the job. For instance, it used to be really easy to get jobs because I knew Python and whatever Python web framework happened to be popular at the time. These days, everyone knows Python, so it's really no help at all.

What's certainly true is that interviews are getting much tougher! Every time I have to interview, I'm fighting against the best and brightest students from all over the world. They're mentally quicker than me, willing to work more overtime, and they cost a lot less. Furthermore, now that Python has become so ubiquitous at Universities, this battle is more fierce than ever. Furthermore, the amount of code companies expect you to be able to write in an interview is getting harder and harder.

On top of all of that, I think the cards are institutionally stacked against more senior engineers. I saw three principal engineers interview at my previous company. Two of them were buddies of mine, so I knew they were really good. That company turned down all three of them.

Some companies have also resorted to using timed programming tests like HackerRank. I like the fact that HackerRank is an objective measurement of a candidate's skills. However, it favors the young since it requires lightning quick thinking and ignores all of the other skills a more seasoned engineer has learned such as testing, design, scalability, requirements gathering, teamwork, agile, code review etiquette, etc.

Furthermore, anytime someone gives me a timed programming test, I completely fall apart. Usually, to be successful, you have to be creative enough to come up with an elegant (i.e. efficient) solution. However, having a gun put to your head is the enemy of creativity. I've been getting better by practicing at LeetCode, but I'll probably never be as good as a bright Russian kid who's been training for programming competitions for years.

The next problem is that the more companies you've worked for, the more hiring managers question whether you're a problematic employee. 2 years at Google right after college? You must be awesome! 2 years at Google, 2 years at Twitter, and 13 other startups? There must be something wrong with you! Forget about the fact that most of those companies don't even exist anymore ;)

The last challenge I'm facing is that the location of my house in Concord means I can really only work at companies in San Francisco or the East Bay. Unfortunately, the only thing harder than commuting to Mountain View (or other parts of Silicon Valley) is trying to buy a house there! Since we're in the middle of adding onto my house, I don't think we'll be moving anytime soon.

In summary, I'm pretty sure I can find my next job. What about after that? Will it keep getting harder and harder as I keep getting older, and there are more companies on my resume and more specialties? What should I do for the rest of my career? I just hit 40, and I'm wondering, am I going to keep on being able to put food on the table when I'm 50 or 60?

Finally, let me express my thanks to all of my friends who have been so supportive lately as I struggle with these questions. My buddies Ryan Larrabure, Alex Martelli, and Jarek Wilkiewicz and all my IronPort co-workers have been surprisingly encouraging. Love you guys!


Vicki said...

I understand the feeling.

I used to be able to say I was a programmer. I liked being a programmer. I did shell, awk, C, and Perl programming. I wrote internal tools for whichever team I was in. I had three Bioinfo company jobs and a position as a utilities programmer for an OS distribution. I loved it. I want to work down the hall (or a quick email away) from my "customers"

But then Perl 6 was announced, companies stopped using Perl 5, those that kept using it wanted Object Oriented module builders... and the WWW brought more web apps and different development and the number of companies looking for the kind of programming I liked doing dropped significantly. These days everyone is a JavaScript programmer. Feh.

So, I slowly switched away from internal programming to internal docs.That's harder, unfortunately, because many managers think the writer should write everything. (Note to managers: if I was the subject matter expert, I'd be writing the code, not the docs. I'm the "turn your docs into readable English" expert.)

Then I got a job doing wiki support. That was the best 5 years of my "career". But no other companies use that wiki, so I have all of this expertise and no one who wants it.

Which is to say, in a round about way, that you're not alone. Good luck to you. Let us know what you figure out... at last for the near term.

Alex Martelli said...

> It seems to me that once you become a consultant as an older programmer, it becomes very hard to get hired as a normal engineer at many companies.

Not in my experience -- close to 50 years old, I was a successful consultant, yet Google was happy enough with me to extend a generous offer, including paying the costs for me and my wife to move half a world away from my native Italy to California, where I've been living since (and still happily working for Google -- as you, JJ, know, but, pointing it out for other readers).

I don't think a few years as a consultant, startup founder, etc, can do any harm to your career, based on my experience...


Jon Seidel said...

I agree with Alex -- going in as a consultant is NOT a dead end. I'm in my 60's and recently got back into programming (loving it!) after many years as a management consultant. I went in as a contractor and within a month and a half I was offered a full-time job. I chose to stay on as a consultant (been there over 18 months now) and everything still is working out. I could convert pretty much anytime if I wanted, but consulting works for me.

I've learned a lot of languages in my day (not so many as you, but I'll bet you don't have IBM/360 assembler language in your list :=) and I do agree that folks want 'specialists" when they're hiring. So in my case, ruby, cucumber, rspec were the key skills that I use every day, but my knowledge of bash especially, plus git, html, javascript are a vital part of the mix.

It definitely took me a while to get back into full-time programming -- bombed on some of the early outings but kept practicing like you're doing and then found what worked. You're so dang good it's only a matter of time before you find the right spot.

Shannon Behrens said...

(Humorously) Thanks to all my friends Hy and Brandon who took the time to read this post and then immediately texted me to point out that I misspelled principal ;)

Shannon Behrens said...

Here's a nice quote from a buddy, "I see your post is getting useful comments ;-) Step 1: be Alex Martelli Step 2: get hired. Yay! Alex is pretty awesome." Agreed!

Shannon Behrens said...

Thank you everyone for all of your useful comments! I'm slowly digesting them. If you're facing a similar situation, you can checkout all the comments for yourself:

Twitter conversation:
G+ conversation:
Facebook conversation:

Shannon Behrens said...

Nice to hear from you, Vicki! There are still a lot of companies that need internal tools. My experience is that they tend to use Python these days, and they're often web apps. I've done that sort of work at multiple companies.

I've always been pretty good at picking up new things to try to keep ahead of the curve. However, being familiar with a technology isn't sufficient these days. They want someone who's used it for multiple years. Since you can only do that for a few technologies at a time, it means you have to pick a specialty. For instance, even though I've used Ruby at several startups, since I'm not a full time Ruby developer, I'm probably not good enough to get a Ruby job. Times have changed.

I'll be writing another blog post with a summary of what I've learned from everyone's comments.

Shannon Behrens said...

Thanks, Alex! That's helpful. I think you're the third person to say that, so I'm convinced.

dan aronson said...

I think that any company that isn't willing to look at the experience holistically as opposed to the language generalist/quick learner is putting themselves at a disadvantage.

Shannon Behrens said...

Dan, I agree. That worked out well for us at our startup. You hired me to code a RoR app even though I had very little RoR experience. I ended up having to learn ActionScript to overhaul the Flash video player.