Master Thesis, Internship

Internship or Master Thesis on How to storage 101: Configuration Space exploration of Modern Linux Storage stacks

Ref. 2024_023

Project description

Modern storage stacks on NVMe are complex and have numerous configuration options that an application can configure. Not only that, to deliver high performance in a reliable and predictable manner, one must understand the intricate interaction between storage hardware, CPU, operating system, memory management, scheduler, and complex hardware and software internals. Storage stacks such as SPDK take many of such roles under their control, skipping the operating system completely. However, modern emerging APIs such as io_uring selectively offer to keep the kernel (thus, maintaining all the benefits of using it - caching, file system, sharing) and still let applications configure many of such performance knobs.

The goal of this work is to build a performance reasoning framework to explore the configurations space of modern storage stacks: specifically SPDK and io_uring. In this context, the key research questions that we want to answer are:

1. Is SPDK style user-space storage stack still the best way to build modern high-performance storage applications? In this context, we would like to benchmark, quantify and explain the performance gap between io_uring and SPDK.
2. Are there design guidelines in which high-performance storage should be integrated in applications. Here we would like to understand how async I/O is integrated and used in state-of-the-practice applications (Databases, key-value store, graph storage, machine learning frameworks).
3. Given a default starting configuration of a machine, can we develop a configuration optimization methodology and implement it to automatically guide the configuration space optimization. Here a system model is needed to be built that can reason about the expected performance, and help us diagnose root cause in case of performance anomalies.

Qualifications:

• Enrolled or in possession of a Master's degree in computer science with a keen interest in data storage research, cloud computing and performance engineering.
• Excellent coding skills: Familiarity with Linux environments and software development tools (git/GitHub, IDEs, gcc, gdb, QEMU, virtual machine and containers etc.).
• High amount of creativity and outstanding problem-solving ability.

Preferred Qualifications:

• Experience with systems programming and internals (kernel, memory management, hardware, CPU)
• Experience with data storage and NVMe storage internals and specification
• Experience in machine learning or cloud
• Experience with performance engineering tools (perf, fio, ebpf)
• Excellent oral and written English with good presentation skills.
• Strong interpersonal skills and excellent written and verbal communication.

Diversity

IBM is committed to

diversity

at the workplace. With us you will find an open, multicultural environment. Excellent flexible working arrangements enable all genders to strike the desired balance between their professional development and their personal lives.

Please submit your application through the link below. This position is available starting immediately or at a later date.