Back in the good old days – and in internet-time, this actually means just a few years ago – people were quite happy with their LAMP stack: Linux, Apache, MySQL and PHP. With this quartet, or a variation on it like PostgreSQL instead of MySQL, we could do everything: create a blog-site, setup an e-commerce web shop, making a guestbook, you name it and it was there..
But times have changed… radically. More and more information is available and must be processed quicker and in more difficult manners than before. Do you accept a web shop where you cannot even do faceting search? Not really. But those systems can be more complex than they appear and lot of components must be used in order to get things working and working fast. In fact, there are so many components that we cannot speak of a LAMP stack anymore. A better name would probably be a LAMPGMVNMCSTRAH stack..
The world according to developers
Is it possible that every developer knows everything about every component or technique currently available? Probably not, but it should be your ultimate goal. Specializing in certain area’s can be a good thing but knowing a lot about one subject makes you indifferent in others. The amount of people out there today that know much about everything is small, very small. Consider it this way: as an army general, you probably can’t drive a tank, shoot a target from miles away or do all the neat commando stuff like Arnold Schwarzenegger does in basically all his movies. Even if a general could do that, there are much better people available for the job: the commando’s, the infantry, the cavalry etc. However, it’s the general’s strength to oversee the whole playing field, to know when to deploy tanks, when and where to send in snipers and when it’s time for Arnie to come and do his job. He must be able to know the strength and weaknesses of everybody, to have at least a certain amount of knowledge about them and most important, how all those small pieces fit together in the big puzzle.
So generals are important. Without them, you will loose the war. But as a soldier, having the correct mentality is just as important. As a programmer, you simply cannot pretend that you and you alone are running a system. You are part of a whole, and even though you might be the most important part, or the controlling part, you still need to work with other components in order to get the job done. If you forget this, no matter how good your part is running, the whole system will fail because you either decided to do too much yourself, or did not correctly utilize the rest of the system.
The world according to system administrators
But being able to design and develop your software isn’t the only thing that needs to be done. Your system must be deployed onto a production infrastructure that can range from a small shared VPS system to a multi-clustered system and everything in between. Maintaining such infrastructure is also getting more difficult but there are lot of tools available like Puppet, Chef and others that can help. With these tools, you basically “develop” your infrastructure just like a software engineers develop software. By maintaining the infrastructure in a versioned manner, deploying new components like CouchDB, Memcache, Redis etc are much easier to achieve because everything can be deployed and tested inside a test environment first. Afterwards, the system can be deployed to many production servers instantly.
Because of the the whole complexity, operations should work closely together with development. Such a new way of doing business is called DevOps, and it rapidly growing in order to deploy more complex systems on more complex infrastructure even faster.
So what’s this LAMPGMVNMCSTRAH-stack? It basically stands for: Linux, Apache, MySQL, PHP, Gearman, Memcached, Varnish, Ngnix, MongoDB, CouchDB, Solr, Tika, Redit, ActiveMQ, Hadoop.. and we have still lots of components we are forgetting (zeromq, rabbitmq, cassandra to name a few).
The world is too complex for one person to comprehend everything and the same goes for software design. It’s sad but true thing though. Being able to know your tools, the way how they interact and to combine them all gives you a big advantage. In the end, set your goal to become a general.