Optimizing Linux: Prioritizing Your Most Critical Applications

To ensure smooth performance and prevent disruptions in mission-critical applications, efficient resource allocation is essential. Linux provides a powerful solution through cgroups and I/O accounting, enabling precise control over resource allocation. In this post, we'll explore how to prioritize applications and optimize Linux performance, while using immudb.io as an example. 

The Challenge

Even with a single mission-critical application running on a Linux machine, other processes can disrupt its performance. File uploads or innocent commands might temporarily starve the main application of essential resources. To avoid this, we need to grant priority access to crucial resources like I/O bandwidth and CPU cycles.

The Solution

Linux's cgroups act as resource containers, allowing us to allocate resources among different processes or users. I/O accounting tracks and controls I/O resource usage, enabling precise allocation. By combining cgroups and I/O accounting, we can reserve specific resources for critical applications, ensuring they receive the necessary I/O and CPU resources for smooth performance.

Leveraging systemd and cgroups

Modern Linux distributions use systemd as the init system, which utilizes cgroups extensively. Instead of resisting this setup, we can cooperate with it by modifying slices and resource allocation. Slices categorize processes, effectively dividing the resource pie for each process.

Embracing cgroup v2

Upgrading to cgroup v2 allows for finer control over I/O resources and more efficient tracking. If your system runs on an older version of Ubuntu or Debian using cgroup v1 by default, follow these steps to switch to cgroup v2:

1. Open the file /etc/default/grub

2. Look for the line that starts with GRUB_CMDLINE_LINUX

3. Add the following text at the end of that line: systemd.unified_cgroup_hierarchy=1

4. Save the file and update the GRUB bootloader configuration with update-grub.

5. Reboot your system to apply the changes.

Customizing Resource Allocation

Focusing on user slices allows us to prioritize applications for specific users. For instance, granting the immudb user a higher I/O weight ensures its processes receive priority treatment. To customize resource allocation, follow these steps:

1. Copy the file /lib/systemd/system/user@.service to /etc/systemd/system.

2. Open the copied file in /etc/systemd/system and find the line that says Delegate=.

3. Modify the Delegate= directive to read: Delegate=pids cpu io.

4. Run the command "sudo systemctl daemon-reload" to update the systemd configuration.

The cgroup Virtual Filesystem

After making these changes, navigate the directory tree at /sys/fs/cgroup to access various files and subdirectories. The user.slice subdirectory contains folders for each user and active session. Running processes are associated with these directories, and their PIDs are recorded in a file named cgroup.procs for each node in the tree.

The file io.weight inside these directories determines the I/O priority for processes mapped to specific levels of the tree. Modifying this value allows us to adjust I/O resource allocation to meet our requirements.

User Resource Management

To fine-tune resource settings for users, utilize systemd's override mechanism. Create specific directories within /etc/systemd/system following the naming convention user-$UID.slice.d, where $UID represents the user's ID you want to tune.

For example, if you wish to optimize resources for a user named "foo" with UID 1001, create the directory /etc/systemd/system/user-1001.slice.d and add a .conf file inside it. The file's content will specify resource settings for that user.

Recent systemd versions (at least 240) support a template directory, user-.slice.d, to manage I/O priorities for all users automatically. However, older systems like Ubuntu 18.04 and Debian Stretch may require creating individual directories for each user.

Conclusion

Strategically managing resources through cgroups and I/O accounting ensures optimal performance for critical applications and shields them from potential interference. By prioritizing your main applications' resource needs, you can create a smoother and more efficient Linux experience. Implement these techniques to fine-tune resource allocation and achieve peak performance for your most critical applications.