Have you ever felt “behind”? Like other people are somehow “ahead” of you and if only you could start again you’d know how to REALLY do things? I thought for sure that I was ahead of the game. I was barrelling forward, certain that I was ticking all the possible boxes. Scholarships? Check. Advanced degrees? Check. Publications in prestigious journals? Check.
Unfortunately, I was in for a rude awakening. I thought ticking the boxes was succeeding. I thought I was winning the game, until I suddenly had to question if I was even playing the right game.
In the last year of my PhD, I realized that academia was not for me. It was almost certain that we would have to relocate and my wife was now pregnant. The thought of raising children away from our parents and extended family sounded horrible. Also, when I was researching post-doc positions, I was looking at the starting salaries and wondering what was up. They were comparable to union construction workers. I could have made the same amount for the last 10 years and not had any student debt?! As I dug in further, I found that 60% of PhDs will not have a job at graduation, 1% of PhDs will get a tenured position, the number of PhDs was increasing and the number of tenured positions was decreasing. I was pretty good, but I don’t like playing games with my life when the deck is stacked so much against me.
Also, I didn’t LIKE the environment. I liked delving into a topic and knowing it inside and out, but I learned it’s expected that professors spend only a third of their time on research. The other two thirds are on teaching (which was really not for me) or administrative work and committees for the university (kill me now). I also wasn’t meeting people I really felt like I connected with. My social group was growing, but it never seemed to contain anyone from my department.
All of this wouldn’t be such a big deal if I hadn’t doubled and tripled down on this career path:
- My undergrad degree was a subject of specialization in physics. I did a Master’s and PhD in theoretical physics.
- I worked my butt off to get the summer research scholarships and invested as much of my time as I could towards publishing academic papers. All of my “jobs” were physics research (other than working on my parent’s farm).
- I avoided any of the networking events as they seemed to get in the way of the “real work”.
Apparently these are all the wrong things to do. Now I had to prove to the world that I was capable of doing anything OTHER than what I had spent the past decade of my life (and tens of thousands of dollars in student loans) doing… with a baby and student loans… sigh…
Fine. This is a solvable problem. People get jobs all the time. What is it that they do? Is it the particular skills they have? Is it something about their resume or the way they get in contact with the people who hire? This is something I can learn. Right?
Wrong. All of my attempts at this went REALLY poorly. My resumes went into a black hole, with no way of connecting with a human to find out what about me needed to be different to get SOME kind of job.
Fortunately, I was able to rely on the one thing I had always been able to rely on: Beer.
During this time when I was trying to find a way to pivot my career, I was making beer with a friend who was also doing his PhD. We would get together every few weeks for the next stage of the creation and bottling and chat about our work. It turned out that his lab was looking to commercialize their research, and they were looking for a team to help grow the technology.
My PhD at this point had involved a lot of numerical modeling and programming, so it looked like I could contribute by being the one to implement the first versions of the software for demos. There was a very good chance that a company would be founded, so it looked like a good opportunity to learn how to make software products involving hardware and potentially join their team as well. Even if it didn’t pan out, it would definitely expand my skills so I could hopefully avoid the resume black hole I had been interacting with before.
So I split my time between trying to frantically learn how to build a software product and ACTUALLY building a software product. I found that there was so much involved with delivering a software product beyond just “How do I do X in this programming language”.
NOTE: If you made it this far and you don’t actually care about writing software, I suggest you skip to part 2.
How to make software
When it comes to learning how to be a programmer, I find that there’s a hierarchy of things to learn when it comes to code:
- Level 1 – The basics of how to make code do things: You start by learning the “How to make the code do X in this language”. Basic control structures (for, if, etc…), memory management, classes, etc… This is what most people think about when they want to “learn to code”. Pick your favourite book, online tutorial, coding academy, etc… whatever helps you pick it up the fastest.
- Level 2 – Advanced topics in that particular language: Every language has tools for doing the thing you are trying to do. Some languages are better at X and some are better at Y. They all have advanced features to try to make your life easier. Programming languages are designed to solve particular problems. I find that those “extras” tend to be the reason for picking one language over another. If you are starting a program from scratch, figure out big things your software needs to accomplish and find a language that is strong in that.
- Level 3 – How to make code do things without a giant mess: You spend 80% of your time reading and understanding the code and 20% of the time writing it. This means that optimizing for reading, understanding and reducing complexity will pay off by at least factor of 4. The topics involved in this sound like nit-picking until you have to work in someone else’s code and you wonder if they were intentionally trying to make the most obtuse piece of crap possible or if that was just a happy by-product. I have found the best resource for this is Steve McConnell’s book Code Complete 2. Read it cover to cover. Twice. I know it’s giant. I don’t care. Read the whole thing. Start with the chapter on functions (Chapter 7) and I dare you to tell me that’s not the most useful thing you’ve learned yet. Start doing code reviews with your team. All those times you thought you were writing gold you will quickly find it’s all in your head. The code reviews are going to feel like big wastes of time. Too many people in a room for too long. Once people stop making the same mistakes over and over again, you’ll see how high leverage code reviews are because you prevented numerous mistakes of the same kind in the future. We are frequently reminded of this when we slack off for a bit and have our own code reviewed and the number of WTFs/minute jumps way up.
- Level 4 – We can’t find any criticism’s about code cleanliness and we’re using the most efficient structures. Why is this still so hard?!: Now you start to get to the design principles. We’ve been learning from Agile Software Development, Principles, Patterns, and Practices on SOLID design principles and a few other books about design patterns (Head First Design Patterns: A Brain-Friendly Guide and Refactoring Guru). SOLID design principles will show you mistakes you are making that are causing things to be hard for you and design patterns will show you the way many of these challenges have already been solved.
How to deliver software
Once you know how to write a piece of code, now you need to manage that code throughout the product’s life-cycle. Follow all of Joel Spolsky’s 12 rules. I would guess that we are an 8 out of 12 right now. Specifically:
- You must use version control. Start with Git and you will be able to work in 90% of teams. You can’t write code with other people without version control. You’re doing dangerous things without a safety net. The fact that version control isn’t a required tool as part of every computer science course shows how out of touch the university is with ACTUALLY writing software. If you want a crash course on “Why” start here.
- You want to reduce the turnaround time, as much as possible, from writing a piece of code to having a build that can be shipped. Any pain points through this will mean that shipping software will always have some fabled “release date” that never comes. The more automated tests and the faster the tests run the better your software quality when you do ship it. This is hard. Ideal workflows are:
- Local commit
- Run as many unit/integration tests as possible to see if anything breaks
- Commit back to main repository
- Run larger scale unit/integration tests to see if anything breaks
- Build the software installer, with all additional tools that need to be pulled together to make a 1 click install
- Host the build so people can get it.
- Learn everything about your development ecosystem. Make sure you know all the shortcuts for your IDE, any third party tools or plugins to make things faster (e.g. integrating with version control, quickdrops, code expansion or formatting, etc…). Leverage every single tool you can get your hands on to accelerate your development. If there are tools that make you more productive, it behooves you to use them.
- Learn how to do unit testing and integration testing. This is a giant topic, which I can’t do justice here but it’s better that you know about it with no explanation than not. Pick an automated test framework for your language and run with it. Two good resources:
- Make it as easy as possible for fixing bugs. We use Bugzilla, but there are certainly more modern tools on the market now. We’ve built into the software the ability to send a bug to our server at any time. Try to automatically grab every single bit of information you can because you will need it when tracking things down. When our system sends a bug, it compiles:
- Software version
- Every single configuration file the software is using
- A query and dump of all of our hardware
- A screenshot of what the user is currently doing
- An error log of the last 1000 internal actions the software performed
- The actual error, if one is present and all recent errors
- A dialog for the user to explain a bit more about what they are doing
How to translate what you are supposed to build into code
On most teams, there is one person responsible for the product. It’s their vision, along with a healthy understanding of the customer’s wants and needs that becomes the thing that will be built.
We’ve had that, but we have also had the programmers be responsible for large portions of the product. There are certainly positives and negatives to this. We had heard about design documents from Joel Spolsky’s blog and we did that poorly for a while until we realized how critical it was. We eventually realized we had to have requirements documents as well. Here’s the path we went through. “Guys, we should build X. It’s pretty simple we just need it to do A, B and C so I don’t think it really needs a design document.” Inevitably, we wouldn’t think about if it was a feature we could use everywhere or just in that specific place. We wouldn’t think about how we could extend the feature or how it could interact with other features. We wouldn’t think about the usability or corner cases.
Andy Grove describes it beautifully: “So why are written reports necessary at all? They obviously can’t provide timely information. What they do is constitute an archive of data, help to validate ad hoc inputs, and catch, in safety net fashion, anything you may have missed. But reports also have another totally different function. As they are formulated and written the author is forced to be more precise than he might be verbally. Hence their value stems from the discipline and the thinking the writer is forced to impose upon himself as he identifies and deals with trouble spots in his presentation. Reports are more a medium of self-discipline than a way to communicate information. Writing the report is important; reading is often not.”
That’s when I realized that the process of putting it all down on paper at least forced the builder to think about what they were building before they got up to their eyeballs. Two people can have a discussion and think they agree but only when there is a screenshot of a UI do you find that they were talking past each other. When it’s concrete, people also start to think about interactions between features that they hadn’t thought about before.
We ended up adding requirements documents, because programmers would change the design later to improve something without realizing it stopped users from doing something else that they could do before. When I asked why the heck it works this way now, they’d say, “It matches the design!” What we realized is that there are MANY user facing designs that could be chosen, but not all of them satisfy all of the requirements. If those requirements are explicitly stated it’s not hard to see what the design needs to do.
At a high level, start with what you think the product/feature must do/satisfy. That goes in the requirements doc. Then start with designs that satisfy those requirements from the user perspective. Then think about how you implement. Our process looks a lot like this now. Hopefully, you have someone responsible for the product who knows the customers guiding the process. If you don’t, I find programmers naturally work in the other direction (what are we capable of building, make a design, does it meet the requirements). They would think about how hard it would be to implement X and quickly the discussion about what the product should do turned into how the code should look. This was always putting the cart before the horse. Decide on what to build first, don’t declare the design should be like X because we can envision how to implement X.
If I was to start out as part of a new boot-strapped start-up today, racing towards revenue before we ran out of cash, I would still do all of those systems in the first year. You may not need your complete continuous integration process in the first 3 months, but you will definitely want your design and requirements docs. Adding unit testing and integration testing will let you build the next stuff WAY faster and having ways of collecting user info will allow you to solve the bugs ASAP without screwing around.
Unfortunately, knowing WHAT do do when writing code is only half the battle. I eventually realized there’s so much more to mastery (Part 2 coming soon).
One reply on “The Struggle to Improve – Part 1”
[…] Part 1 of this series, we spoke about some of the struggles transitioning from academia and learning the […]