I will participate in Google Summer of Code (GSoC) 2024 with a project titled “Improve Rust benchmark suite.” The official coding period started a few days ago, so I’d like to introduce my project.
Announcing Google Summer of Code 2024 selected projects
What’s GSoC?
GSoC is a program focused on bringing new contributors into OSS development. This year marks the 20th anniversary, and it’s the first time the Rust Foundation is participating.
I feel like it’s a huge challenge for me because this is my first time1 contributing to OSS. I’m working on a large project, and most of the communication is in English, which is different from my mother tongue. Additionally, since the academic semester in Japan starts in April, most of the project period is not during summer vacation. Anyway, I’ll do my best :)
If I finish the project, I might write another article about applying to GSoC.
About the Project
rust-lang/rustc-perf is a project that collects performance benchmarks for the Rust compiler, rustc. It’s composed of two crates: the collector for collecting benchmarks and the site for the website that visualizes them.
In my project, I’ll improve the rustc-perf project, especially the site crate. Here are some high-priority tasks:
-
Embed query trace visualizer into rust-lang.org (RLO)
- Currently, we use Perfetto for visualizing query traces. However, Perfetto doesn’t provide a way to embed it into the website and only supports deep links. As a result, it takes a long time to load, sometimes fails, and requires users to navigate between multiple pages.
- In this task, we’ll aim to embed a query trace visualizer into RLO using tools like Catapult or the Firefox Profiler.
- Related issues:
-
Add runtime benchmark graphs
- We collect both compile-time and runtime benchmarks in
rustc-perf. The runtime benchmarks are relatively new and aren’t as well visualized as the compile-time benchmarks. - This task involves adding graphs for runtime performance on some pages, achieving the same level of visualization as compile-time performance.
- Related issues:
- We collect both compile-time and runtime benchmarks in
These are just examples, and we might prioritize other tasks in the future, but currently, I’m working on these two. Additionally, I plan to make minor improvements along the way.
Current Progress
I’ve only had a few days to work on this project due to other commitments, but I’ve tried some query trace visualizers, such as Perfetto, speedscope, and Catapult.
The (minimum) requirements for the visualizer are:
- Can be embedded into the website
- This requirement ruled out Perfetto. Although embedding Perfetto might be possible with significant effort, I prioritized other options first.
- Reasonable speed
- speedscope’s interface was excellent, but it took over 5 minutes to load rustc’s 300,000+ record query trace JSON, often causing a stack overflow. Using protobuf might help, but I decided to try other visualizers first.
- (Should) not depend on anything other than HTML, CSS, and JavaScript
- Among the options tried, Catapult showed the best results. While Catapult is not actively maintained and many projects have moved to Perfetto, we think it’s acceptable for
rustc-perfand decided to try it. - Catapult has a “trace2html” program that generates standalone HTML of query trace visualizer. In the draft PR, I used this program to generate HTML files, which the server then reads and returns as a response. However, this program runs on Python, which isn’t the best, so I’ll try the Firefox Profiler next. If that doesn’t work, we’ll just use Catapult.
- Among the options tried, Catapult showed the best results. While Catapult is not actively maintained and many projects have moved to Perfetto, we think it’s acceptable for
I also made a PR to update npm packages used in rustc-perf (which slightly changed the colors in Highcharts.)
Conclusion
I’ll (hopefully) keep posting updates on my project and GSoC 2024. (I’ve been switching between ChatGPT, DeepL, and an English dictionary again and again to write this post…) Also, feel free to contact me on Zulip!
Footnotes
-
Except for fixing typos ↩