o1js API Redesign

[Trivo25]

This issue serves as a central discussion point as we transition into the implementation phase and begin rolling out the new API incrementally. It outlines the core philosophy and design approach behind the changes and will be shared both internally and externally. Feedback and discussion are encouraged - feel free to contribute your thoughts.

Problems with the Current API

The current abstractions, particularly SmartContract, provide a smooth onboarding experience but introduce challenges when building complex applications that align with Mina’s zkApps protocol architecture. Many protocol features are hidden behind high-level APIs that don’t always map well to their underlying functionality.

This abstraction makes it difficult for developers to fully grasp the system’s inner workings, limiting transparency and control. Additionally, fine-grained settings and transaction structures are often inaccessible, preventing developers from fully leveraging the zkApps protocol’s capabilities. As a result, o1js can become a bottleneck rather than an enabler for advanced zkApp development.

Proposed New API (WIP – Feedback Appreciated)

Note: This section primarily outlines external-facing API changes. Some internal improvements and refactoring of the types and bindings layer will also be made, but these are purely technical updates necessary to support the new API.

The new API introduces the following features, each designed with a specific goal in mind:

Improved Transaction Creation and AccountUpdate Composition

Similar to the existing Mina.transaction block, this new API will integrate with the rest of the updated API. It provides developers with explicit tools to construct transactions, send them to the network, and manually create complex AccountUpdate structures. This ensures greater flexibility and control over transaction logic.

Local Transaction Simulation

The core transaction logic will be fully translated in o1js, granting developers more control over their local environment. This will be accompanied by a testing framework that allows developers to fine-tune network state, debug transactions more effectively, and build more robust zkApps as well as better understand how zkApp transactions are applied to the network.

MinaProgram

A new API representing a program running on Mina, replacing SmartContract with a more flexible and transparent interface. This change enables developers to express application logic in a more direct manner while maintaining full control over on-chain account structures, transactions, AccountUpdate composition, permissions, update logic, and assertions - without unnecessary high-level abstractions.

ZkStateMachine (Reducer Interface)

A zkProgram that defines a state machine and its state transition function. This will replace the existing actions and reducers abstraction, providing a more structured and powerful way to integrate actions and reducers into MinaProgram and zkApps.

ZkBuffer (Accumulating Side Effects)

Designed to work alongside ZkStateMachine, actions & reducers, the ZkBuffer serves as an on-chain buffer that accumulates side effects (e.g., payouts, token transfers, state updates) for later execution. This addresses one of the key limitations of the current API.

ZkEnum

A zk-friendly, easy-to-use Enum implementation. This will simplify the implementation of action and reducer logic while also improving the handling of side effects through ZkStateMachine and ZkBuffer.

Goals of the New API

• Transparency: A lower-level, transparent API that empowers developers by providing a clearer understanding of the system
• Readability: The implementation should be easy to follow, with code-generated documentation that allows users to seamlessly navigate between the documentation and the source code
• Flexibility: The API should not enforce rigid workflows, as developers will have diverse and unpredictable needs when building zkApps
• Modularity: The design should support extensibility, allowing higher-level abstractions to be built on top of the core, making advanced functionalities more accessible over time
• Safety: The API should minimize potential pitfalls by design, preventing common developer errors

Explicit Non-Goals

• Simplicity: While avoiding unnecessary complexity is important, the core API should prioritize transparency over simplification. Higher-level abstractions can be developed on top of the core to provide a more user-friendly experience, but the foundational layer must remain clear and explicit rather than overly abstracted

Next Steps

This thread will serve as a space for discussion, where we gather feedback, exchange ideas, and compile information for future improvements.

[microbecode]

A zkProgram that defines a state machine and its state transition function. This will replace the existing actions and reducers abstraction, providing a more structured and powerful way to integrate actions and reducers into MinaProgram and zkApps.

I'm confused. What's the difference between MinaProgram and zkApp? I've usually seen zkApp refer to on-chain apps. Maybe you meant to say "reducers into MinaProgram and ZkProgram"?

Otherwise, I love the rename of SmartContract to MinaProgram - makes it clearer that these live on-chain, and that ZkProgram is the other type.

[L-as]

I think you should consider having a more functional API for interaction with Mina too. I don't mean for circuit generation, that would be very hard, but e.g. sending funds, updating state, etc. It's already functional under the hood, yet the API abstracts this and gives you a mutable API that sets fields and then returns them for you.

While this is perhaps easier for beginners, I've come to believe that it's worse in the long run.

Perhaps this is already something you've considered.

[kadirchan]

Safety: The API should minimize potential pitfalls by design, preventing common developer errors

I think some of the current approach of o1js to developers is make things difficult especially for beginners. It would be better if .get() methods handles fetching more automatically rather than this erorrs.

[dfstio]

Now, when Zeko has removed the number of AccountUpdates limit, and I hope Mina will do the same during the hardfork, we will have txs with many AccountUpdates. I have two suggestions in this regard:

• Add getAndRequireUpdated() function that will return the last state written on blockchain, similar to getAndRequireEquals() or, if this state was updated in the previous AccountUpdates in the same tx, the updated state. Right now we have requireEquals() and need to calculate manually and provide to the contract the value of the state variable in case it was updated in the previous AccountUpdates.
• Add the optional argument to tx.prove(accountUpdates: number[]) to be able to prove the tx in parallel on several workers, each worker proving some subset of the AccountUpdates. For example, proving the tx with 200 AccountUpdates with proof authorization on M2 Max now takes 17 min. We have a clear use case here: DEX

[Trivo25]

A zkProgram that defines a state machine and its state transition function. This will replace the existing actions and reducers abstraction, providing a more structured and powerful way to integrate actions and reducers into MinaProgram and zkApps.

I'm confused. What's the difference between MinaProgram and zkApp? I've usually seen zkApp refer to on-chain apps. Maybe you meant to say "reducers into MinaProgram and ZkProgram"?

Otherwise, I love the rename of SmartContract to MinaProgram - makes it clearer that these live on-chain, and that ZkProgram is the other type.

Good point! I guess this hasn't been very clear in the past. Here's how I see it: your zkApp (consisting of an on-chain part and an off-chain part, eg. the UI) could consist of one or multiple MinaPrograms - so a zkApp is the whole package, the finished product - I hope that makes sense!

[Trivo25]

I think you should consider having a more functional API for interaction with Mina too. I don't mean for circuit generation, that would be very hard, but e.g. sending funds, updating state, etc. It's already functional under the hood, yet the API abstracts this and gives you a mutable API that sets fields and then returns them for you.

While this is perhaps easier for beginners, I've come to believe that it's worse in the long run.

Perhaps this is already something you've considered.

Definitely agree with this, parts of the new API will be more functional in how they allow you to define transactions and AccountUpdates - I will share an update once we get closer to releasing parts of the new API and then we can have a more detailed discussion

[Trivo25]

Now, when Zeko has removed the number of AccountUpdates limit, and I hope Mina will do the same during the hardfork, we will have txs with many AccountUpdates. I have two suggestions in this regard:

* Add `getAndRequireUpdated()` function that will return the last state written on blockchain, similar to `getAndRequireEquals()` or, if this state was updated in the previous AccountUpdates in the same tx, the updated state. Right now we have `requireEquals()` and need to calculate manually and provide to the contract the value of the state variable in case it was updated in the previous AccountUpdates.

Very good point, this hasn't been easy in the past. The plan is to make the local state cache more reliable as well and giving developers more control over it (eg. setting the local state to a specific state) - this will not only make sending and anticipating chain state easier, but also provide the foundation for a better test suite.

* Add the optional argument to `tx.prove(accountUpdates: number[])` to be able to prove the tx in parallel on several workers, each worker proving some subset of the AccountUpdates. For example, proving the tx with 200 AccountUpdates with proof authorization on M2 Max now takes 17 min. We have a clear use case here: DEX

Would these workers be locally, integrated into o1js, or should o1js provide an interface that developers can build on top of and provide their parallelization solutions?

[Trivo25]

Safety: The API should minimize potential pitfalls by design, preventing common developer errors

I think some of the current approach of o1js to developers is make things difficult especially for beginners. It would be better if .get() methods handles fetching more automatically rather than this erorrs.

Yes! Hopefully with the local transaction application logic and better state cache and fetching, this will be more transparent as well

[dfstio]

Would these workers be locally, integrated into o1js, or should o1js provide an interface that developers can build on top of and provide their parallelization solutions?

It makes little sense for workers to work locally as o1js is already optimized for parallel threads, so the workflow would be to serialize the tx, send serialized tx to several separate workers in the cloud to prepare proofs, receive the proofs, and add them to the tx before sending to the Mina node.