"First Year" retrospective blog post #65
Comments
|
First high-level outline here: https://docs.google.com/document/d/14iRikCmQF39Y_gq2TEc8OOL-uHofiaFCzEoeFmFMLFo/edit Feel free to make edit suggestions and add comments in Google Docs ^^^ |
|
Maybe as guidance, here's an example of a great Annual Report of a FOSS project: https://openssl-corporation.org/post/2025-02-14-annual-report24/ : Recognition of contributions, open communications about funding, clear strategic direction -- just the amount of color used may be debatable (sun glasses do help :). |
|
I'm not sure who's on the content review committee, but I would remind you that the Technical Charters for both PQCP and OQS say that each project's TSC has responsibility for "coordinating any marketing, events, or communications regarding the Project", so please ensure that any parts of the blog post about the projects are reviewed by the corresponding TSCs. OQS was not contacted prior to the PQCA/NVIDIA cuPQC announcement in January, which I reminded @hartm about after I saw the public announcement. |
|
Quick reminder: The issue @dstebila alludes to above is not that that OQS has no representative on the content review committee (it does) but that PQCA on several occasions did not bother to involve it. See below for a full timeline.
If Spencer is also willing to spend the time (?), more eyes make for better postings. But first order of the day going forward should be to agree to inform the group when PQCA does announcements pertaining to any of its projects: OQS learned about this PQCA activity from an external source and had to scramble to align reality with announcement. In my personal opinion, this was bad style eroding the level of trust between PQCA and the communities it controls. Full timeline for fact checking: I do understand that commercial interests (in this case, PQCA to promote LinuxFoundation & NVIDIA to promote its product) drive such announcements, but wouldn't it be appropriate to at least inform the affected FOSS community about them ahead of time? Better, even let them have a say/fact-check things? Wasn't this the whole point of having this "content review committee"? The only thing in the above that makes me wonder: @dstebila is quoted in the PQCA announcement on Dec 3, so arguably was aware of it (correct, Douglas?) ... So was that the reason why PQCA did not inform the wider OQS community/TSC? I do understand and acknowledge (IBM/LF consider) OQS as Douglas' project but the actual work falls onto more shoulders -- and things like this are anathema to more participation as it seems commercial interests trump those of the FOSS community. |
I'd been asked by NVIDIA for a quote in September 2024 which I provided, but wasn't further contacted regarding the PQCA announcement in January 2025. |
|
Yes, sorry about this, I'd (obviously incorrectly) assumed OQS was aware of the NVIDIA PR process and had provided feedback. I agree that better formalizing this review process would be a good idea. One cautionary point: I'd like to make sure that we can move quickly on things like blog posts and get approval/feedback in a relatively short amount of time. Good posts can help us get attention and attract contributors, so it's in our interest to incentivize people to post and write. A long approval process, particularly one that folks might view as adversarial, would not help us here. Thank you all for your time! |
|
We will make sure that the different TSCs: OQS, PQCP, and Tooling sign off on the content before publishing. Since @SWilson4 @planetf1 and @n1ckl0sk0rtge (leaders of the different TSCs) will be writing the content for the appropriate sections, we are asking that they will get 👍 from TSC. |
|
We discussed this at the PQCA TAC just now. Here's a concrete proposal for future posts: We ensure that all posts go through a PQCA content review committee, and we ensure that each project has representation on the committee. The review process will be fast (2/3 days) and focused on factual errors. I am going to push the governing board hard to put together the outreach committee, and we can formalize this process as a part of the outreach committee's work. I will note that most LF projects handle this kind of thing through the outreach committee. Having each TSC approve when they only meet every other week just doesn't scale. We are going to frustrate people if we don't put their blog posts out quickly or if the review process is onerous, and asking potentially every TSC member to read and vote on a blog post is a recipe for slowing the process down. |
This "process" existed and was not used. All it had taken was a single email, 15 seconds worth of effort. The development effort of cuPQC has surely been measured in years, not days, let alone seconds, right? So "speed" cannot have been the reason for this.
I understand your urge to move quickly for marketing reasons, @hartm. I also completely agree with your statement that "Good posts can help": Quality is key -- particularly in security software projects. So, in sum I think we could agree that due diligence is essential for security software, right? If so, do we further think people seriously qualified in this space will contribute because of "quickly released" blog posts teeming with "business-oriented" exaggerations? I don't and actually think the contrary is true: Experts in the field are actually repulsed when seeing that a security software project is more "quick marketing" than actual delivery.
Let me state the obvious: Doing (serious) security software is frustrating and tedious. If you change the process to write about it to be "quick" (i.e., let business-oriented people in an "outreach committee" decide by fiat what works), this sounds like a surefire way to publish promises that do not hold water if looked at carefully by the people having to deliver (just maintainers and committers, not all, particularly not technically active, TSC members!) and depend on those promises (users). All told, I think that was is proposed here is an approach working against the interests of the community of FOSS developers and users favoring solely the interests of marketing teams. |
I agree with you!
The blog post you mention here was neither quickly released nor teeming with business exaggerations. You objected to a word like "ironclad" describing security and had a quibble on the definition of utility. Those are very minor things, and people use this kind of wordplay even in serious academic papers. In fact, one of my favorite papers on hardware-based functional encryption is called "Iron"! I made those changes because you'd asked for them, but if I'd been a potential contributor who wasn't heavily involved in the project, I would have been seriously put off by that process and those comments. I think we all have the goal to get more contributors to come to the project. We also don't want to overburden our contributors with review work for things like blog posts. So it's important to find a balance here. An extra adjective in a blog post is probably not going to be the root cause of a major attack.
The proposal was to have a content review committee consisting of, at a minimum, technical representatives from all projects, approve content from the outreach committee. This committee would even be a part of the outreach committee to ensure everyone is on the same page. I'm not sure how this is different than what you're advocating above? Can you explain what you'd like to see (if it's different from the above)? In a dream world, all of the best developers and technical people would write lots of content in layman's terms explaining what they were doing, major milestones, and everything else you might imagine. But this never happens (not in PQCA or any other major open source project of which I am aware). Technical folks are busy, and they prefer writing code to writing blog posts or other media. When they do write blogs or other things, they want to get them done quickly and get back to technical work. But we are mostly forced to be reliant on marketing or other people writing about the projects. In this vein, if you have any suggestions as to how we can get more technical people writing on the blog, I'd love to hear them. We could really use more content and it could be a valuable tool to attract more contributors. |
I do see a difference in an industry association publicly announcing a "move towards ironclad production code" and a research team using the moniker "IRON" to name their system, don't you, too? This, in conjunction with the use of the term "minor things" feels like I'm being belittled or ridiculed.
This is a laudable goal but imo cannot serve as an explanation why contributors that offered to do reviews are not getting involved to do so.
This contains a grain of truth, but again is not a generally true statement holding for all "technical folks": There are technical people aware of the effect that blog posts or other media have on their work, the expectations thus created and demands towards them raised, don't you think too, @hartm?
Doesn't this make it even more important to give the people affected in their work by what is written a chance to voice their concerns in a way free of pressure or ridicule? |
Using "ironclad" instead of "secure" is a very minor thing. It doesn't affect the substance of the blog post at all, and if we put all blog post contributors through this level of scrutiny over word choice, we're never going to have any second-time contributors. I have no idea why you think you're being ridiculed or belittled. That being said, I think both of us do have much more important things to do than discuss synonymous adjectives in a blog post.
Almost all technical folks that I talk to tend to view even marketing-oriented materials about their projects in a positive way, as they can help drive attention to the project and potentially even aid the project in gaining users or contributors, even if they aren't absolutely precise technically. Most people understand the fluid nature of open source development, and no one is looking to hold individual developers accountable for general plans in blog posts. Project roadmaps change, too: should developers of open source projects be forced to deliver on a roadmap if lots of developers leave the project? Of course not; open source is voluntary contribution. And it certainly does not make sense to hold a blog post to a higher standard than an official project roadmap.
The most important thing here is to figure out an acceptable way to formalize this process going forward with the outreach committee, and you didn't address this at all.
Is this plan acceptable? If not, what would you prefer to see? A precise proposal, if the above isn't satisfactory, would be wonderful. If you address anything in a response to this post, can you please be sure to include an answer to this? Thanks! |
Correct. The thing is that we discussed the truth content of the post, though: The blog post announced/promised something that I saw as not attainable with the given code and project setup. That's what I discussed about and you fixed, not just any "synonymous adjective".
Please consider that these generalizations do not fit all people('s understanding).
Well, the thing here is that I'm not a normal developer. I carry responsibility for the code. I'm its maintainer. I have written large swathes of it. People do infer some obligation with that -- among other things, to not simply walk away from it: This may be arcane for you working for companies that change business strategy sometimes every quarter, re-allocating "resources" "suitably", but I do think continued responsibility to deliver on equally responsibly formulated goals is important for the long-term success of FOSS projects, particularly in the area of security software. Thinking some more of it, may it be this (difference in) feeling of responsibility to be the root cause for all our mutual misunderstandings, @hartm?
I thought I did:
This plan is not just acceptable, IMO it already has been executed: This "content review committee consisting of, at a minimum, technical representatives from all projects," already exists, doesn't it? Again, timeline:
I may not understand the concept of the separate "outreach committee" (pointers to term, function or setup welcome!) but other than that, what's missing -- short of PQCA involving it? If you address anything in a response to this post, can you please be sure to include an explanation for this -- or where I'm off with my understanding of your question? Thanks! |
Yes, I agree totally. This is the root cause of the misunderstandings here. I'm glad we pointed it out in the clear. I admire your dedication and devotion to secure code. It's a fantastic quality for a developer to have! But I will caution you that most developers and companies view responsibilities with respect to media as I represent it here, and their expectations (for both responsibilities and for putting out media pieces) will be set accordingly. Also, remind me to tell you some horror stories about press releases in a social setting. Some are pretty unbelievable.
We are putting together an outreach committee to handle PQCA marketing. This is common in open-source projects, and ensures there is coordination between different companies and project members. We are just going to roll the content review committee in to the outreach committee when it's formed so that it will be directly tied into marketing and PRs. Hopefully this will streamline everything here. Thanks again for your time! |
This is not an answer to the repeated questions by @dstebila and myself:
and its "corollary":
I'm not sure adding yet another PQCA committee will make things easier. To the contrary, I'm concerned it will probably siphon off even more time and resources direly missing in development, thus making it even less likely to bring back features OQS has lost since joining PQCA.
So it seems the net of this is that you are saying that my quality/responsibility demands --at least wrt to writing publicly about it-- are higher than those of PQCA, right? This to me personally would be a massive problem: Before joining PQCA, OQS (not just myself, but many good responsible people) has worked over years to establish a reputation of responsibly developed software and equally honest external representation of it -- and the attitude shown above runs counter to (being able to maintain) that. OQS always acting responsibly in all regards was arguably a key reason why some companies --even against openly documented advice-- use this security software in production. Thus, by lowering the bar wrt responsible behaviour PQCA (if the above indeed is the overall organization's perspective) not only creates new risks to those companies but also taints the reputation of people working on the project and it becomes understandable why taking a distance from it is preferable to contributing. This reputation of OQS and the people working on it arguably also was what LinuxFoundation and IBM wanted to make use of when creating PQCA.I do not recall either representative, @hartm nor @maximilien, in the discussions leading up the take-over of OQS into PQCA ever mention the need for OQS participants to lower the bar on responsible behaviour, though: Hearing this on the "one year anniversary" is a bummer to me personally. But it indeed makes me understand finally why we had so many misunderstandings during this time. This also offers an explanation now why I began to feel attraction towards other FOSS projects: They simply may be more in line with my understanding of responsible behaviour. In consequence, let me ask the following questions:
|
|
I'm happy to answer your questions and I also thought they were understood.
We thought the project was informed.
I am not sure. It slipped through the cracks. We certainly weren't trying to be deceitful about it. We could do a postmortem, but Kenny doesn't work for the LF any more. That being said, I will promise that we at PQCA are more diligent about following the content release policies in the future. We will not let something like this slip through the cracks again.
The goal in adding the outreach committee is to encourage new contributors and users. Since the composition of the outreach committee will not be developers, I'm not sure how it would siphon time from developers, unless you're worried about the time spent reviewing outreach content.
There are two different issues you bring up here (I think) with your quality/responsibility demands. The first seems to be related to guarantees of the project. As an open-source project, we cannot force contributors to work. If the maintainers change their minds about doing something, we (the LF, or PQCA) can't do anything about it. That is by design; the maintainers are supposed to have power here. Therefore, we (again as the PQCA or LF) are fully aware we can't make promises with 100% certainty, and anyone who understands how open source works gets this. We always represent things honestly, but have to accept that it may not be totally accurate. We are not a corporation that can force delivery of a roadmap. If individuals or companies want to make promises about support, that's great. They can do that, but it's on them. To go back to your previous assertion on this:
People do walk away from code they maintain all the time. You yourself told Douglas and I last May that you'd be walking away on no notice due to struggling with the CVE process on this. While we're certainly glad you only ended up walking away for a little bit and not leaving permanently, we would hope that you would be sympathetic to people leaving from time to time for all kinds of reasons (startup folds, job changes, etc.) and the potential issues this causes in open source software. The beauty of open source software, though, is that, unlike closed code, if you rely on code and developers stop maintaining it, you can pick it up yourself. The second seems to be related to the accuracy of "external representations" of things, or media. Marketing people simplify concepts so that non-experts can understand, and that there is a point where things go too far in pursuit of exactness at the expense of understanding. Technically speaking, there probably is no "secure" cryptographic library of any size, because there are always bugs in any code, and there are basically no cryptographic libraries that have not or will not be permanently bug free. But does this mean libraries shouldn't market themselves as secure? Of course not, and many (OpenSSL, Signal, WhatsApp, etc.) do just this! Saying "our library probably isn't absolutely secure but no one is likely to find a bug because they are really hard to find and if bugs are found they are usually by white-hats who properly report them and we put in a lot of effort to make it secure so you can probably use it safely" might reflect the truth of every cryptographic library better than just saying "secure", but it would also confuse a lot of users. I think it's OK to have some imprecision in a marketing post design for non-experts if it makes it easier to understand the content. Even serious academic papers simplify things often to the point of incorrectness in their introductions and technical overviews, where the same principle applies. Going back to the adjective, using a word like "ironclad" with respect to security doesn't mean that something will be 100% secure (ironclads sink sometimes!), just that we'd be putting in a lot of effort to make things secure. I think that's an accurate representation of all the great work that you and others do for OQS.
I'm not sure what you think I'm proposing would cause a massive reputation loss.
I don't understand this comment at all. First, how are we "lowering the bar"? Second, you just argued about how the OQS community was special because there was a commitment to (I'm paraphrasing) long-term secure development. Now you're saying somehow PQCA is tainting the reputation of people and they should not be worried about that long-term secure development principle? Sorry for my lack of understanding your argument here.
When did we suggest lowering the bar on responsible behavior? I'm not sure why you're inferring that. It is true that OQS and the cryptographers behind it, certainly including but not limited to Douglas and Michele, have fantastic reputations. But most software developers probably can't name a single cryptographer, and even security experts can probably only name those that have Turing awards. You're welcome to ask IBM on this, but I suspect that the biggest reputation that they (and the other companies that founded the PQCA at the LF) wanted to capitalize on with the formation of the PQCA was the reputation of the LF in the general developer community, which is large and overwhelmingly positive.
I'm not sure how you got to this conclusion. My takeaway from this is that I'm going to work with the community on the year-end blog post, and we will be 100% certain this time to follow all of the review procedures. |
The TAC would like to see a 1 year anniversary blog post for PQCA that summarizes what we have learned and accomplished.
This is being targeted for March release.
Each project and working group should plan on contributing to the TAC's 1st year blog post and then follow up with individual "the year ahead" blogs afterwards.
This issue is specifically for tracking the 1st year blog post. The individual projects/groups should open Issues in their own repos if they feel it is appropriate.
All content will be reviewed by the content review committee https://lists.pqca.org/g/content-review
The text was updated successfully, but these errors were encountered: