The Bliss of Automating my Server Setup

I have a VPS, on which a few of my side-projects are hosted. Recently, I had to set it up a few times, because I made it unusable (please don’t ask me how, it was really stupid).

Every time this happened, it took me hours to set it up again. Granted, I don’t know much about maintaining a server, and you can probably do it a lot faster than me, but you can never be faster than a script. So, after the second time, I finally decided it is time to automate that process.

Goals

I’m hosting ElasticSearch and a few .Net Core applications. They are all accessible via an nginx reverse proxy.
Additionally, there are a few cronjobs.

While writing the script, I decided to add Docker support to all my .Net Core applications, so there is one less thing needing to be installed.
This is extremely easy btw, just right-click -> Add Docker support -> Done. But I digress.

The script should

  • install Docker, docker-compose and nginx
  • pull all Docker images
  • copy the nginx and ElasticSearch configuration files to the right locations
  • add cronjobs
  • start all applications
  • configure the server so that after rebooting the applications start automatically
  • install other applications that I like (e.g. htop)

The Script

Writing the script was simple, quite a lot simpler than I thought it would be. It was basically copy&pasting the commands from the vendor websites and using the Docker CLI, which is extremely well documented.

Docker: https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository
docker-compose: https://docs.docker.com/compose/install/
nginx: https://www.nginx.com/resources/admin-guide/installing-nginx-open-source/#prebuilt

I did have some minor difficulties with specific software though, which are described in the following 2 sections.

ElasticSearch

ElasticSearch has a small fallacy. By default, the vm.max_map_count environment variable is set to 65536.
This is too low for ES, which results in out of memory exceptions. This is very confusing, if there are a few GB of free memory.
The solution is setting vm.max_map_count to at least 262144, as mentioned in the docs, which is quickly found after some googling.

Cronjobs

Sadly, crontab does not offer a way to simply add an entry. But as expected, others had the same problem, and Stackoverflow has an easy to use solution for it.

(crontab -l 2>/dev/null; echo "@reboot nginx -c /home/ubuntu/nginx/nginx.conf") | crontab -

This also demonstrates how applications start after a server reboot. Simply with an @reboot crontab entry.

Conclusion

Automating the setup process has brought down the time until my server is ready from hours to minutes. And in those blissful minutes, I don’t have to do anything, just watch the script work.

Additionally, I feel much more comfortable experimenting. If something breaks, I just have to start the script.

Automation is great!

Automate all the Things

PS: If you are interested in the whole script, you can find it here, minus my credentials of course ;)