I remember the days when I’d develop using simple Linux command line tools. When I worked at Amazon almost 10 years ago I used a lot of old-school tools like Vim, ssh, and grep. It took some time to get familiar with them, but I figured them out just by reading a manual page or watching my coworkers. For better or worse, developer and ops tools are getting so complex we have to read books to become proficient. Newer tools offer more features and scalability, but it comes at the expense of complexity to learn and manage. How should we decide when to stick with simple tools and when to invest in more powerful or complex ones?
Complexity Creep
The search for development bliss has made our toolchains more complex. Instead of using a simple terminal text editor like Vim, now developers use ones like Atom or VS Code that include hundreds of modules. Package managers like NPM pull hundreds or thousands of transitive dependencies of often unknown origin. Instead of writing JavaScript directly, now we write in dozens of languages from ES6 to CoffeeScript to C++, all transpiled back to JS.
On the infrastructure side, the cloud software revolution freed us from managing hardware, but now we manage an even more complex set of microservices and orchestration tools. Microservices run inside frameworks… on app servers… which run inside containers… on Kubernetes… in the cloud… configured by terraform. Monitoring these services is no easy task either. Instead of using simple text files for logs, solutions like Elastic Stack shard data across a cluster of log servers and agents trace requests across distributed nodes.
This is just a small slice of the technologies developers need to learn. Take a look at this cloud native landscape from the Cloud Native Computing Foundation (CNCF).
© 2018 Cloud Native Computing Foundation. All rights reserved. CNCF Cloud Native Interactive LandscapeThis is impressive, but can you imagine being a beginner and having to learn all these technologies? It’s enough to make your head spin! Every solution you add to your stack requires extra time to set up and maintain, extra time to learn and train employees, and extra mental space. You have to spend days comparing the pros and cons of different solutions, and when something goes wrong, you have to deal with all of that complexity to fix it. It makes one yearn for the days of simpler tools.
Innovators Should Value Agility
Too often I see startups adopting the latest buzzword technology, whether it’s a new JavaScript framework on the front end or an orchestration framework on the back end. Usually these new technologies are more complex and introduce more risk than the less exciting and mature ones.
The vast majority of new products don’t need enterprise-scale infrastructure right away. Using overly complex solutions too early can lead to technical debt as your needs change. Creating a minimum viable product (MVP) isn’t the only way to stay lean; also consider your minimum viable infrastructure. Invest in solutions that are simple initially, and swap them out as your needs change.
The most important thing startups and innovators need to be successful is AGILITY. If you are still reaching product market fit or pivoting your solution, you need to quickly validate the hypotheses for your product with minimal waste or overhead. The needs for your project and architecture will inevitably change as you adapt to market needs. To be agile, you need tools that allow for fast iteration and that can be quickly adopted and changed when needed.
Use the Right Tool for the Job
When you don’t need complexity, take advantage of simpler dev tools. If I have to make a quick text edit, I’ll use simple and fast text editor like Vim. If I’m diving into a complex project, I’ll take the time to load the more complex VS Code or Atom.
When starting a new project, think about using a managed platform like Heroku so infrastructure is out of the way. Setting up an EC2 server and load balancer might not seem hard, but now imagine automating deployments several times per day with CI/CD, hosting different versions, and scaling it, and you see how complexity starts to add up.
Also, keep your monitoring tools simple so your ops team isn’t bogged down setting them up. Tools like Splunk and Elastic Stack are powerful, but take more time to set up on a cluster including the agents, the search indexers, and the front-end UI.
When you’re starting a new project, Papertrail™ is an easy logging solution to use. It handles log management for you in the cloud so you don’t need to manage a log server. You don’t need to install agents, and you’ll be done typically in 45 seconds. Its live tail mode looks like what you’d see in your Linux terminal—and it works nearly as fast!
Simplicity in your dev tools and infrastructure can clear space in your brain. Don’t waste too much time on fancy tools early on. Save your brain space for things that really matter to your business, like building a better product and helping your customers.