In my new role I’ve been diving into a dearth of new technologies. Each and every day is like scaling a new mountain of complexity. Some technologies my new team uses:
- A functional programming language for the .NET platform
- Containerization technology
- A streaming technology
- A 3rd party continuous integration tool
- A general purpose Object Oriented programming language
- Multiple database technologies
- A caching technology
- Machine Learning technology
That is just one of two stacks I am in the process of learning to work on! I seem to find a new technology every day!
I took some intentional time today to reflect on my newfound realization that nothing is simple anymore. And what I realized is that although I can have an intelligent conversation about what each of these technologies does and why they are needed, most of the time I can’t tell you why someone decided that we needed these technologies in the first place. For instance, I know why a streaming service like Kafka is a great idea for certain applications but I don’t know the exact circumstances it was conceived under because I wasn’t there. Redis is a nifty technology when latency is really an issue. But up until this point I had never worked on something where that type of latency needed to be addressed. I know why Continuous Integration is a good idea but it really isn’t that old of an idea. Where was I when someone was deciding that it was a good idea? In all honesty I was working on something that a lot of people were just realizing was a bad idea!
I know that for me this is really not an easy admission to make. In general I make it my business to consistently ask “Why?” before I ask “How do I?“. This strategy has always served me well when I had the time to practice it. However, in my ramp up in my new role I’ve realized that in order to add value quickly I have to learn to use a technology before I ask where it came from. I fully intend to backfill my understanding but that will come in time. But one thing that sticks out to me as I work to understand these new technologies is that even if I asked “Why?” for my whole career, I would never be able to gather enough information to say exactly how and why we ended up where we are now! It is increasingly obvious that the pace of software innovation is accelerating. If this is objective fact, then it follows that the more that time passes, the more unlikely it is that anyone has a complete picture of how we got where we are. Literally: the better we know, the less we know!
The real problem I am getting at that those who are in the earlier stages of their career just don’t have enough context to have a complete picture of why things are done in the ways that they are. In my own personal experience, I started out my career working on an older monolithic application. The process of learning how to work on that system was advancing my knowledge. However, the more time I spent learning that old and outdated way of doing things, the less time I spent on learning something new and valuable. The more time I spent on it, the further it left me from where I wanted to go. I was actually going backwards.
In my job search over the past year I ran into many instances where there was an implicit expectation that I knew certain things by default just because I was a software engineer. A particular example was the expectation that I knew the concepts associated with distributed systems. In these instances my interviewers just seemed to assume that I would know these concepts in the scenarios I was given to solve. Why is this? In these scenarios I was asked how I would build systems that would scale to millions of data points being processed at once, be fault tolerant, use particular database strategies, use caches, and employ load balancers. How is one supposed to gain this expertise without having worked on a system that uses these components before?
But then the question is how do we know what we don’t know? I had this question answered for me as I interviewed and found out what employers value. In the process I was able to identify what I lacked and rectify the situation by learning new things in my own time. But what struck me is that once I actually got on with a company doing distributed systems, the nuances and challenges of the art became something I could relate to. Being around them and working on them accelerated my learning so much more than just reading a book about them. Once I was given the opportunity to work on distributed systems in a real life setting my learning and skillset accelerated. I found that I picked things up quite effectively in a reasonable amount of time. It makes me wonder why we are so aggressive in selecting for prior experience in a technical domain like this when all it takes to learn the skills is an opportunity?
Be honest with yourself when you answer these questions about your career, wherever you might be in its course:
- Do you know who said this was the way things should be done?
- Do you know what happened to inform the way things were being done?
- Do you know when it was said that this is the way things should be done?
- Do you know where things were headed when it was decided that this is the way things should be done?
- Do you know why this was the way things were done?
I would say that it is not a good strategy to incriminate those who do not know the answers to these questions day one off the bat. Likewise I would say that it is also not a good strategy to incriminate yourself for the same. None of us had all the answers when we started. But I’d encourage you to utilize these simple English words: who, what, when, where, and why, to orient yourself firmly in the domain of Software Engineering. I’ve found the practice to be very beneficial. Knowing, or at least pursuing their answers leads us further down the road than, say, just picking up the latest version of a technology. People these days seem so obsessed with what skills one has but can rarely answer all of these simple questions at once no matter where one is on the competence continuum. What would be the result if we all knew the answers to these questions all at the same time?
Maybe you were luckier than me and started out doing large scale distributed systems day one. Maybe you came out of school and were right at the edge of the curve in a domain. But I suspect that there are many out there like me who just took a few years to get to the cutting edge. And in the end I think that is just fine.
In the end most of us have to work for a living. Thus we value technical skills for the impact that they can bring to the businesses we work for. But it is not only this that matters. I got into technology because I love the idea of breaking new frontiers. I think that those who do break frontiers can add far more value than just business value. We all seem to worship the idea of starting the next billion dollar company. But what about the next billion dollar technical achievement that changes the game? What about the next Iphone, the next networking protocol, the next GPU, the next machine learning technique, or the next language?
What do you value the most?
When you dream of writing the best software of your life do you dream of what it will do for a business?
Or would you say its the best by what it does for people who use it?
I know what I value.
Striving so hard over this past year in my horrific job search has seemingly landed me right at the current edge of technology.
I think I earned it.
I want to create a new edge.
Are you at the edge? Do you know who, what, when, where, and why?
Maybe if we asked these questions more then we would all have a clearer picture of what is valuable. By using them to orient ourselves in the field we might make make revolutionary changes in our own skillsets. And finally, by articulating their answers we might derive a new vision for the next evolution, or even the next revolution, in technology.
Maybe if we all had some answers we could push the edge one step further.
Now that I’m at the edge that’s exactly what I’m going to do.
I am also an aspiring Twitter mob leader. Join me!