On the shape of the Rust project
Various thoughts after the Rust Leadership Summit and Rust Nation
I had an exciting week in London attending the first Rust Leadership Summit and Rust Nation 2024. It was great meeting all my fellow contributors and team members again in person.
For the first time, project leadership got together to discuss the big open questions that face the Rust project today - how we organise the project and plan our work.
Since becoming compiler team co-lead last year, I’ve been thinking about these questions in the context of the compiler team. I wrote about roadmaps and planning in On ongoing work in the compiler team, and team structure in On compiler team membership (part I and part II).
After the summit, I’ve refined my own personal vision for how I think the Rust project should be structured. Not everyone in the project would agree with this vision, and all I can do is make a case for it to my fellow project leads and members, while also listening to my colleagues in the project and learning from their feedback and critiques.
Much of this vision is inspired by the discussions we had at the summit, so I want to be clear that not every idea, insight or framing in this post is my own.
One of the priorities of the project’s leadership council1 is determining the structure of the Rust project. This involves answering questions like “what is the team structure of the Rust project?” and “who are members of the Rust project?” - there’s a council committee to work this all out.
These are important questions, as the Rust Foundation dials up its support of the project, with various grants to contributors (e.g. hardware stipends, travel to conferences, etc), then it starts to matter more who is considered a member of the project.
It isn’t just a recent concern, there was a Rust All Hands event planned in Thessaloniki for early 2020. You might have guessed that it was a certain pandemic that nipped that plan in the bud, but it wasn’t, the project had grown so much that paying for everyone to attend became infeasible (this was when Mozilla was still the primary supporter of the project and most contributors were volunteers).
What is the current structure of the project?
There are a lot of teams in the project at the moment and over 258 people2. I’ve always found that number surprisingly big3.
There are lots of different groups in our current org structure, which can be roughly organised into the following categories:
Implementation teams/working groups
Teams which perform implementation and bug-fixing for some subset of the toolchain.
e.g. t-compiler, t-rustfmt, t-rustdoc, wg-diagnostics
Gatekeeping teams
Teams that are responsible for reviewing RFCs and proposals for some subset of the toolchain.
e.g. t-lang, t-libs-api
Maintenance teams
Teams that are responsible for making sure that critical bugs are fixed, performance doesn’t regress, backports are reviewed, etc. for some subset of the toolchain.
e.g. t-compiler, t-libs
Short-term working groups/projects
Groups organised around pushing forward the design, implementation and eventual stabilisation of an individual feature or set of features.
e.g. wg-rfc-2229, wg-polymorphisation, wg-nll
Support teams
Teams which are necessary for the successful day-to-day operation of the project but don’t directly contribute to the project’s deliverables.
e.g. t-moderation, t-infra
Umbrella teams
Teams which don’t really have much active work on their own, and exist to group teams together in the org chart.
e.g. t-dev-tools
Other teams
Teams which don’t directly contribute to the project’s deliverables and perform some other function (e.g. community engagement).
e.g. t-community, twir
Defunct teams/working groups
Teams which used to be active and perform some function but don’t really do much anymore and still exist on the org chart.
e.g. t-community, wg-self-profile
User working groups
Groups of users interested in Rust for a specific domain, created earlier in the project’s history for a handful of domains.
e.g. wg-embedded, wg-gamedev
These categories aren’t perfect, the compiler team does a little bit of gatekeeping too when we decide which compiler flags to stabilise, but that isn’t the bulk of our work.
What is the purpose of the Rust project?
When deciding what the shape and structure of the project should be, I think the most important and first question to think about is around the purpose of the project - what is our ultimate goal?
During the leadership summit, there appeared to be two contrasting visions for the project4:
For some, the project should be focused solely on delivery of the Rust toolchain (the language reference/specification, compiler, standard library and dev tools), and as such the project should be comprised solely of the teams that do this work (and those teams necessary to support them, like the infrastructure and moderation teams).
For others, the project should be about more than just the toolchain, and include other teams and groups, such as a community team and user groups (like our current embedded or game development working groups).
Personally, I’m pretty firmly in the first camp - a smaller Rust project. I’d love to understand the arguments of those who want the Rust project to do more better, so please do reach out if that’s you.
There are a handful of reasons why I think we’d be better off with a smaller Rust project:
Back when the Rust project’s world domination success wasn’t as clear, it made more sense for the project to create community resources, host events and spread the good word. In 2024, there are a lot more people using Rust, and it would be healthier for the Rust project to not be the centre of the world and to trust the Rust community (or Rust communities) to grow and do their own thing. The project has done a good job at seeding the initial community culture and norms - things like having codes of conduct and caring about the safety of your projects - and now we can let it flourish on its own.
It’s also important to focus on doing one thing well - delivering the best possible Rust toolchain. We have limited resources in the Rust project - whether that be funding from the Rust foundation or bandwidth in project leadership - and we shouldn’t spread ourselves too thin. There’s a lot going on just with the design and implementation of Rust, never mind anything else.
I’ll grant that there are risks associated with moving toward a smaller project: we would need to be careful and thoughtful when moving some groups out of the project as users in these groups value being part of the project and we want them to continue to succeed and thrive. It’s also possible that this could lead to members of the Rust project becoming more isolated from the community and Rust’s users (though I think that is less likely).
It’s also important that there is still support for community groups, something like a Rust Society as suggested by Jonathan Pallant in a recent blog post, and from the Foundation.
What do teams do?
Teams are currently organised around different parts of the project - the compiler, the standard library, the language’s design, etc. Not all teams do the same tasks though, how the team works is largely defined by the nature of the part of the project they work on.
There are five functions teams can serve in the project. We don’t currently distinguish between these functions of our current teams, which can lead to unfilled niches and confusion.
Design
Design teams would take problems that Rust users have and invent solutions which would then be reviewed by “gatekeeping” teams.
There aren’t any design teams in the project at the moment5.
Gatekeeping6
Gatekeeping teams are responsible for reviewing incoming proposals and changes to the language, standard library, etc.
e.g. t-lang, t-libs-api
Implementation
Implementation teams take accepted features and RFCs and implement them in their part of the project.
e.g. t-compiler
Maintenance
Maintenance teams maintain the quality of their part of the project - ensuring that critical bugs are fixed, backports are reviewed, performance doesn’t regress, there is appropriate review capacity/latency, etc.
e.g. t-compiler, t-libs
Support
Support teams aren’t involved in building the toolchain (the Rust language’s product), but do vital work to keep the project going and support the other teams.
e.g. t-infra, t-moderation, t-release
In addition to these “hard” functions, there are “soft” functions that teams perform, such as conferring recognition and status to its members. It’s important that we have mechanisms for highlighting the great work that contributors are doing and the expertise they have in the project and language, but this can be in tension with the other functions of the team7.
Does the project need a design team?
Design teams don’t currently exist in the project. Contributors who are good at this type of work are rare - it’s often very challenging, requiring a deep understanding of the language’s semantics and what makes something Rust-y.
As a project, we want to support projects like Rust for Linux, but there’s nowhere in the Rust project to take their requirements and work out the best solution for them8.
Project members who want to do this work don’t quite fit into any of our existing teams, and as a project, we don’t know what to do with someone who doesn’t fit into the team structure.
Why make a distinction between implementation and maintenance teams?
In general, we don’t currently make a distinction between implementation teams and maintenance teams, but we’re starting to do this in the compiler team with RFC 3599.
Conflating implementation and maintenance causes issues because these are in tension:
We want a large implementation team, anyone who has contributed for a while and we trust should have permissions to review and approve other contributors’ pull requests. There is always a lot of work to do on the compiler - bugs to be fixed, refactors to be performed and features to be implemented. We only ever get more contributions to the project, and having more reviewers is necessary to keep the project sustainable for the existing contributors.
We want a smaller maintenance team - maintenance activities mostly involves decision-making. Anything that involves decision-making benefits from having a smaller group that can reach consensus easier. However, it is important that group represents a diverse range of opinions and perspectives. Historically, keeping the number of people involved in final comment periods manageable has been the reason for the distinction between the compiler contributor and compiler team member roles.
What if you could be a project member without being a team member?
We’ve always conflated project membership with team membership - everyone who is a member of the project today is a member of some team, sub-team or working group. What if we didn’t do this?
Implementation teams are different from the other functions described above as these teams aren’t responsible or accountable for anything. It’s obvious that the work of implementors is vital - it pushes the whole project forward - but whether to accept an RFC, approve a backport, or ensure performance doesn’t regress has always been the responsibility of smaller teams (t-lang) or subsets of larger teams (t-libs-api, t-compiler members). Today, many teams have adopted “contributor” sub-teams, reflecting the common need to recognise and grant permissions to contributors without impacting the team’s decision-making.
If teams were only their gatekeeping or maintenance functions, and all implementors were instead “project members” rather than “team members”, then this would resolve some of the tensions between wanting a large pool of regular contributors who are empowered to contribute (i.e. having merge permissions and participating in reviews) and wanting a small team making decisions.
Traditionally, we’ve asked that contributors only use their permissions and do reviews to those subsets of the project that their team is responsible for. I’m not sure that this is necessary. In the compiler, nobody knows the whole codebase, it’s just too large and complex. We trust our team members and contributors to review and approve only what they’re comfortable with9, and I don’t see why that wouldn’t scale to the project as a whole. I think compiler team reviewers could also review pull requests to the standard library and vice-versa, because we trust those reviewers to re-assign when they aren’t comfortable reviewing a change, and that could just as easily apply to rustdoc, rustfmt, clippy, etc.
In my own contributions, I’ve definitely wanted to work on library features that I was motivated to push over the finish-line, but felt that I should stick with compiler work because that was “my part of the project”10. For those members of gatekeeping teams, who want to move away from that type of work and focus on other things, there’s no process for remaining a project member but unaffiliated with a team.
Other large open source projects are a lot more liberal when they give out permissions than the Rust project. LLVM has ~1800 contributors with merge privileges (~1000 after proposed activity requirements), that’s a lot more than we do. After you’ve got “a track record of submitting high-quality patches”, you can ask for commit access.
We’ve been making it easier to join the compiler team in our recent RFC, recognising that to keep up with the patches we’re receiving and the work that team members are doing, the team has to grow.
There is a risk in having easier-to-obtain, project-wide permissions, just look at the recent xz backdoor, but I think that a culture of reviews, quick backports, and having a lot more people involved can mitigate this risk11.
What is a team when there is project membership?
Once we start to think about the possibility of being a project member without being a team member, there are a lot of possibilities for how we could structure the project.
I think it’s important that teams have clearly defined remits and responsibilities and a good way to conceptualise that is to have the leadership council have ultimate responsibility for everything in the project, and then delegate subsets of that to individual teams. For example, the leadership council would delegate responsibility and accountability for the compiler to the compiler team.
We could do this using team charters12 - it would be great to be explicit about what decisions each team makes; what each team is accountable and responsible for; and what each team is consulted and informed about.
Teams could further delegate subsets of their responsibilities to sub-teams. For example, consider the types team, who would inherit responsibilities and accountability for a subset of the language specification and compiler implementation from the language and compiler teams respectively.
Teams would exist to perform gatekeeping and maintenance functions for a subset of the project - such as the compiler, standard library or language. Project members particularly specialised and interested in an area could join a team to participate in those gatekeeping and maintenance activities. Team members wouldn’t need to stay on a team forever, they could rotate in and out as their interests change. Team leadership would be elected from team or project membership13 and would be responsible for ensuring the team is fulfilling its chartered purpose14.
Contributors could ask any team lead to become a project member after establishing have a track record of contribution15. A team lead would sanity-check this track record and check with other team leads for any concerns before making the contributor a member of the project, when they would be granted merge permissions, infrastructure access and added to the review rotation.
Project members would move to alumni status when there has been no contribution activity for some period of time, and can become project members again upon request once becoming active again. Given that project membership may involve with financial support/benefits for maintainers from the Foundation then it’s important that members are active to be eligible.
A risk of this formulation is that project members don’t have a clear team lead anymore that is responsible for “their part of the project”. Without an obvious team lead, there could be diffusion of responsibility with regard to giving encouragement, recognition and feedback to contributors.
In this conceptualisation, the definition of a team is a group that has been delegated accountability and responsibility for some subset of the project - a group that isn’t explicitly delegated these in a charter is not a team. In other words, teams are strictly focused on the maintenance and gatekeeping functions for their subset of the project. As such, teams would be much smaller, just as the teams with this sort-of purpose are today (e.g. t-lang and t-libs-api).
Teams would aim to have just as many people as are interested and necessary to perform their chartered function effectively and efficiently. Compared to individual project members, team members would take a more project-focused perspective, acting in the interests of the team and project, and being invested in the health of the project and the team’s remit.
However, a risk of having smaller teams is that it removes a mechanism that the project currently has to recognise individual contributors and their achievements, and that there could reduce the incentive to join these teams.
What about the leadership council?
Council representatives would still be elected from team members, but it would open the possibility up for different schemes too - a fixed number of representatives elected from all project members, or a mix of team representatives and project member representatives.
Team membership might select for those project members who have more interest in administrative/organisational work and who would be good candidates for council membership.
Does it make sense to separate teams further into maintenance and gatekeeping teams?
Gatekeeping and maintenance teams serve distinct functions and may be better suited to different contributors - approving backports is a different kind of decision than approving an RFC, and requires a different skillset.
Given that project membership is effectively an “implementation team” for all of the project, another possibility would be to have a t-maintenance which performs these activities project-wide, and have per-project-area gatekeeping teams.
What about umbrella teams?
I’m not personally a fan of having umbrella teams that just exist to group together other teams, and would instead opt to have many more top-level teams (e.g. t-rustdoc, t-rustfmt), and just allow for the possibility of teams to choose not to be represented in the council.
It is important that every project member has a council representative that they can speak to about the issues being discussed in the council, so any team which opted not to have a council representative themselves would need some other representation somewhere16. This applies to project members without a team affiliation too.
What about design teams?
Design teams don’t exist in the project at the moment, but it is an important function that we should have capacity for.
There are two possibilities for how this could be implemented into the project structure:
Establish a design team with interested project members who have the experience necessary for this type of work. This team would speak to users (such as the Rust for Linux project), understand their problems and design solutions which would then be proposed to the other project teams.
Acknowledge that this is something that project members (regardless of team affiliation) can do. It would still make sense to have a dedicated space for project members interested in this work to gather and discuss design, but it means that the teams that exist in the project all have a clear part of the product’s deliverable that they are responsible/accountable for, which a design team wouldn’t have.
What about support teams?
Support teams, like infrastructure and moderation, are different than other teams. In both, membership cannot be as straightforward as in other parts of the project due to the sensitive nature of their work (e.g. security and privacy implications). Wherever possible, these teams should try to unify their processes with the rest of the project, but there will necessarily be divergence.
What about working groups?
The difference between working groups/projects/initiatives and teams/sub-teams is that teams and sub-teams have responsibilities and work that they are accountable for17. There should be a clear distinction between permanent parts of the project structure and ephemeral groups that are spun up and spun down to push projects forward. Teams and sub-teams are permanent parts of the project structure and working groups are ephemeral.
Participating in a working group should be sufficient to consider yourself a member of that working group, regardless of whether or not we track that, but working group membership shouldn’t imply project membership.
What if there were levels of project membership?
Another possibility enabled by general project membership would be recognising the “soft power/influence” that some project members have given their tenure and significant track record of contribution18. It’s unclear how exactly this could be implemented, there are lots of open questions (how it would be decided, etc), but it making this explicit could be interesting. They could be called “Distinguished Project Members” or “Rust Fellows” or something.
How much independence would teams have?
Currently in the project, teams have a lot of freedom to define their own processes, up to and including things like what chat platform they use19. Cross-team collaboration suffers when team members aren’t familiar with the different workflows and processes of other teams.
It would make sense to have a unified flowchart for “how to contribute to Rust” that all teams are stakeholders of rather than having individual workflows for contributing to each of the individual parts of the project.
Everyone having a broad knowledge of the project’s processes makes more sense if most project members are unaffiliated with a team.
Other differences in team processes, such as how membership works and what chat platform is used, should be standardised as much as possible.
However, it is important to acknowledge that there is a risk that unifying processes results in teams working less efficiently (if their current processes were well-tailored to their workflows) and being unhappy with having less independence.
Does the project need an operations team?
In addition to the existing support teams, there is the opportunity to create a new support team that assists with the administrative and organisational tasks of other teams - an operations team.
Operations team members would assist project members and teams with meeting notes, agenda preparation, organising and structuring information, noticing dependencies between work happening in different teams and maintaining project documentation.
One of the most important functions that a operations team could perform, in my opinion, would be to maintain a “project handbook”, a single place where all project/team documentation, charters and processes would go. We have far too many disparate sources of information - repositories for every individual team, working group, initiative, project; and the Rust forge, etc. Other than developer guides, everything should be in one place.
Well, that was a lot. Thanks for reading.
Many thanks to Tyler Mandry for organising the summit, and also to Nicholas Matsakis, Mara Bos, Josh Gould, Erik Holk, Jan David Nose, Jakob Degen, Wesley Wiser, Santiago Pastorino, Josh Triplett and everyone else who attended for their thoughts and opinions during the summit and conference.
I’m not a member of the leadership council, to avoid any confusion, just interested in these topics.
There’s 258 people according to Eric Huss’ visualisation, but 435 people in the rust-lang/team repository - I don’t know which is right.
If you asked me to estimate the number of people in the project, I wouldn’t get close to that number. In the compiler team, we have maybe forty people regularly contributing in some way - and the compiler team is comfortably the largest team in the project! We interact most with the language and library teams, and when I think about the number of people in each of those teams, I can probably think of around ten or less. Even when I think about teams that I have even less regular interaction with, like some of the developer tools teams, the infrastructure team or the moderation team, then those teams are either a handful of people or similar to the language and library teams. I end up with estimates of around a hundred or so people, less than half the official number!
I’m sure that there are project members who find themselves somewhere in the middle of these two visions for the project’s structure as well.
While language and library team members do often propose solutions to their and users’ problems, they’re doing that in their capacity as individual contributor, as it isn’t a responsibility of their team to design features.
Gatekeeping as a word can have negative connotations - nobody wants to be a gatekeeper, right? I don’t intend to use the word in that sense, there’s a necessary role in the project for some team to be responsible for what APIs get added to the standard library and RFCs get approved and added to the language, and to make sure that those align with the project’s values - that’s inherently gatekeeping, but the good kind.
I considered calling this “decision-making” instead, but I also describe maintenance teams as doing decision-making and that would be confusing, then I considered “design approval/review” teams, but when I talk about “design” teams later then that could be confusing too, so gatekeeping it is.
Conferring status and recognition through team membership tends to incentivise making it harder to join a team, which is in tension with wanting to grow the team to keep things sustainable (implementation and reviews being spread across more people).
Rust for Linux contributors will sometimes propose fixes, but don’t always have all the context necessary to come up with the “right” solution and they certainly don’t have time to do that for everything they need - they are implementing Rust for Linux after all.
In the compiler team, I like to encourage reviewers to be on the review rotation for the entire codebase. Lots of small trivial patches get submitted that anyone can review, and it’s helpful to have a larger reviewer pool handling these small patches. These smaller patches help build up experience in different parts of the project too. We trust our reviewers to know when a patch needs more experience than they have and to re-assign when appropriate.
I'm not sure how many other contributors feel this way. I know that we’ve been seeing some contributors become part of multiple contributor teams recently.
Reviews can’t entirely mitigate this risk, there’s always going to be work to do to improve supply-chain security, but it’s a trade-off, there are many benefits to having enabling a larger contributor pool.
There are some charters in the rust-lang/governance repository which never took off, and we should revive something like this.
I think it’s important that teams have two co-leads, to distribute the work that team leadership involves, and be able to respond to team members in different timezones. Team leaders should rotate out on a staggered schedule, so that other project members have opportunities to enter into project leadership and implement fresh ideas.
This is both doing the concrete work that the team’s charter entails but also ensuring that the team has enough members to do all of that work.
Exactly what “a track record of contribution” means would need to be decided. In the compiler team’s recent RFC, we described this as follows:
It is [team membership] explicitly intended to be granted more liberally to contributors who have demonstrated competence and trustworthiness, for whom they would be able to work more effectively with these permissions and can be trusted to use them responsibly.
Thanks to Jonathan Pallant for making this point!
While working groups/projects/initiatives may have some output that they are working towards, they ultimately aren’t responsible or accountable for it - that remains with their parent team.
People like Niko Matsakis, Mara Bos or Eddy Burtescu.
With the exception of t-rustup and some of the user working groups, every team has standardised on Zulip.