Figure title (preferably 1280x720 pixels)

Force processes to use swap in memory

Force processes to swap memory (Ubuntu 18.04) When benchmarking processes, looking how a process perform when swapping memory can be useful. Since the operating system determines the memory allocation policy, testing with limited memory can be cumbersome. One could try to fill the virtual memory with other processes to limit the available memory but since the operating system tries to swap inactive processes, our benchmarked process will probably remain in virtual memory. In any case, we cannot know this for certain. ...

March 2020 · Bart Cox
Figure title (preferably 1280x720 pixels)

Minimizing idleness in Spark Clusters

The GDelt database 1, on 18-11-2019, consists of 492,618 segments. Processing the top 10 most mentioned topics for each date on the whole data set would take a long time. Using clusters in the cloud, like AWS EMR, significantly decreases up the needed computation time but might be costly. To make the best use of the clusters on AWS EMR a minimization of the idle time of the machines is desired. ...