Posts

Showing posts from November, 2018

DIY cluster: hardware [Autumn 2017]

Image
Following graduation, one of the first projects I set myself was to design and build a cluster. The motivation was that during my masters project, in an effort to produce an original thesis, I had inadvertently glanced off a much more general problem. Namely to do with the decomposition and parallelization of tasks. At a conceptual level, this is pretty simple. You just break down whatever process you want to do concurrently into blocks, determine the dependencies between those blocks, then allocate them to functional elements as appropriate. These "functional elements" can in principal be anything, humans in a team, servers in a rack etc. Simple, right? The devil of course, as always, is in the details. Engineering concurrent software is hard. Engineering systems that can handle arbitrary concurrent software is very hard. Which brings me neatly to this project. I wanted to build a system which I could use to get a feel for the sort of problems that arise when writing, using