revisit the "RFC Acceptance" policy

[nikomatsakis]

The "async rfcbot decision procedure" was initially meant to be:

• Somebody does "rfcbot merge" or whatever, creating a checklist
• Everybody else is actively monitoring their GH notifications and so they immediately respond, checking their boxes (or raising concerns)
• We promptly move into FCP, which lasts for 10 days; during this period, you get notification from a wider group beyond the team
• At the end of FCP, presuming team still agrees, we go forward

But here is what actually happens

• Somebody does "rfcbot merge" or whatever, creating a checklist
• A few people from team check their boxes, but the rest don't notice it in the overall deluge of GH notifications
• Meanwhile, the wider community is only following a few RFCs and so they notice the comment immediately, and conversation starts
• Some weeks or months or even years later, you finally convince the final team members to check their box
• Then we wait 10 days for FCP, which are either completely quiet, or filled with people rehashing the same arguments that were raised before

I think we can do better.

[nikomatsakis]

Observation:

• In practice, we are often tempted to skip the FCP when things seem uncontroversial.
• When things are controversial, the FCP serves little value.

[Mark-Simulacrum]

I would like to also observe that I (among others) have frequently wanted to "spawn" FCP for (usually) the library team, though sometimes compiler team too. This is not something that I've ever wanted to do for the lang team I think, as they usually have somehow "weightier" decisions, but I imagine it could come up in practice too.

I have heard pushback to this idea, primarily with the general sense of "teams are overloaded as is" -- I think I'm mostly sympathetic, but I also see that with the current (actual) strategy for dealing with rfcbot, there's not much gain beyond potentially avoiding spuriously pinging the whole team.

Not really any concrete thoughts here, just wanted to get this down on here as well :)

[BatmanAoD]

Idea:

"final comment period" would start as soon as the checkboxes are generated, not once they're all checked. A link to a Zulip or Discord thread would be provided for the actual final discussion; new comments in the GitHub thread would be limited (via moderation, I suppose) to unique arguments, with some way of tracking which team members share which concerns. (I think the bot itself may already support this?) All auxiliary discussion would be pushed to Zulip/Discord.

That may not be practical, but perhaps it inspires some other ideas.

[Manishearth]

Yeah, I've advocated for having a more diverse set of venues for RFC discussions in the past: it's worth considering if an assigned chat channel would be useful for the whole lifetime of the RFC, actually. Can be temporary and archived if we're worried about that.

[nikomatsakis]

On a related note, and in the spirit of tossing out thoughts:

Overall, for complex discussions, I would like to get more in the habit of having a more formal notion of collecting concerns and responding to them. I have thought about e.g. FCP being more of a "one way" thing, where people post their comments to some mailing list and then we publish them at the end, summarized. I don't think that necessarily makes sense sometimes but are also problematic and prone to turning into run-away fires.

In my idealized world, perhaps, we'd host a complex discussion like so:

• There'd be some period (say, a week?) of discussion
• We'd pause, collect what was said, and post a summary at the top of a new thread
• We repeat, but now we can observe if the summary is changing, further we can be more aggressive about moderating out repeat comments that just seem to be repeating points that were made there
• This gives a very concrete notion of when "steady state" has been reached
• FCP is just one of these rounds, maybe?

Perhaps you could do the same thing, but not based on time, but rather total number of comments. You could easily imagine automating this by having a bot that freezes a thread after N comments and pings the team (or, better yet, the shepherd / liaison of the RFC)

[Manishearth]

I believe this was kind of the point of summary comments, but those cause tension as well if posted by someone not perceived to be acting in a non-partisan way.

I really like having summaries though!

[Ixrec]

I am almost convinced that some kind of semi-formal summarization process is necessary (albeit not sufficient) to improve the signal/noise ratio on RFC discussions. "Increasing participation" and "more diverse venues" are admirable goals, but if pursued before signal/noise improvements I fear they'd only increase redundancy and confusion, making everyone feel less involved rather than more.

I believe the impartiality tension can be sufficiently mitigated by:

• having a designated role responsible for producing these summaries (presumably the RFC's shepherd)
• not deleting the older stages of the discussion (merely deemphasizing them)
• explicitly stating in process documents that "disputing the summary" is perfectly legitimate (if done civilly as always), since there's plenty of ways a summary could end up missing or misrepresenting something without anyone acting in bad faith
• ensuring whatever platform we do RFCs on allows summaries to be edited (and if we do aggressively prune duplicate comments, maybe we'd also prune disputations that the summarizer agreed with and incorporated)

[nikomatsakis]

I believe this was kind of the point of summary comments, but those cause tension as well if posted by someone not perceived to be acting in a non-partisan way.

Yes. I think summarizing is an important job. I actually think it makes sense for them to be somewhat partisan, though. That is, I think the summary should reflect the thinking of the decision makers -- but they should include the counterarguments. The idea being "these are the arguments we considered and why (thus far) we've decided against them". This can help people to either come up with fresh arguments, or argument specifically against the counter-arguments -- and at some point I think get moderated out as "sorry, we consdiered that, we're moving on".

In any case, I think the "fresh comment thread that begins with a set of summaries" is kind of a key addition, vs "summary comment is lost somewhere in a spiel of comments".

Also, "take a breather while summarization is happening".

i sort of like the version that is triggered by raw number of comments, since it means that smaller RFCs can continue unperturbed.

Anyway, this is perhaps- a more expansive proposal I am making here, I would also imagine that we could do more narrow changes to begin with -- e.g. just starting the FCP from the point of "fcp merge", effectively.

[XAMPPRocky]

On the topic of starting FCP from fcp merge, what happens when the fcp concludes? Obviously if everyone checks their box the RFC is merged. What if one or two people haven't checked their box? What about if a majority of people haven't checked?

[nikomatsakis]

@XAMPPRocky I imagine it would be this:

• FCP concludes when both of these have happened:
  • time has elapsed
  • sufficient boxes are checked (currently: up to 2 may be unchecked, iirc)

[XAMPPRocky]

That would solve the additional wait period that we currently have, though I'm not sure it would help solve RFCs going into limbo. I wouldn't want to have an immediate time pressure to decide once the time is elapsed as I feel that would generate tension between members. But maybe once the time has elapsed and there aren't enough checked boxes, the RFC could be nominated for the team to discuss to merge/close at their next meeting.

[Manishearth]

@Ixrec heh i've proposed something similar in https://manishearth.github.io/blog/2019/02/04/rust-governance-scaling-empathy/

but yeah, it's not necessary to make it a different group of people, i just find that it's easier to avoid the conflict that arises when people see their arguments characterized by the person who is disagreeing with them.

[nikomatsakis]

So, this is a small targeted change, but I think we should do the following:

• Eliminate pre-FCP, as described earlier, so that FCP begins immediately, but doesn't end until all of the following have occurred:
  • At least 10 days have elapsed.
  • Sufficient team members have checked their boxes.
  • There are no outstanding concerns.
• Tweak the FCP close/postpone commands so that they don't require checkboxes, but instead begin an immediate FCP, and allow for the possibility of blocking concerns being raised and resolved.

[nikomatsakis]

I opened rust-lang/rfcbot-rs#288