Codex Update Jan 29nd - Feb 5th
The Codex team continues to make progress with various initiatives to wrap up the demo for the Q1/Q2 public testnet release. An internal testnet has been running for the past few weeks and has been used to test the latest version of Codex and can be accessed using the Codex Testnet Starter documentation.
Ongoing and new lines of research and development will soon begin in preparation for the next version of Codex that will be used for the mainnet release.. Here are the updates from different team members and their ongoing work. Development is currently broken into three distinct teams:
- Client, Testing, and Infrastructure
- Marketplace
- Research
The different teams have actively moving on various fronts. The following are their team updates to various ongoing Epics.
Client
Epic: Nim Improvements
Completed:
- Filed issue for adding Atlas / non-Nimble support for packages
- Start working on Atlas command changes
Ongoing:
- Create a repo as a place to start implementing some core async-threading tools for Chronos like worker pool & disk io on top of the ThreadSignalPtr primitive
- plans to support refc & orc
Epic: Wiring the Proving System
Completed:
- Wrapped ark-circom in a C FFI via:
Ongoing:
Epic: Improve Client Stability
Completed:
- Updated profiler branch for debugging
- Ported the profiler to Chronos V4
- Wrote separate test runner for two client test to try to figure out the origin of a file size bug which magically disappeared
Ongoing:
- Finish work to take down draft flag from PR Expiry per dataset
- Write tests for PR Safe block deletion (with ref count)
- Look into the CI/docker packaging/local build tooling for Waku and Nimbus as part of build improvements PR
- Chronos V4 branch
- Pinned versions for Chronos and json-rpc
Marketplace
Epic: End-to-end Testing
Completed:
- Rebased multinode integration test refactor which had two failing tests due to the erasure coding changes
- Rebased Marketplace integration test suite
- Added support for Result types using formatIt for logging proxy
- Finished the verifier contract
- Deployed a dummy verifier on local networks for testing
- Finished updates to nim-ethers, all tests passing, including in Nim v2
- Fixed an issue in the nim-ethers json-rpc update
- Derived Signers could not access the derived getAddress and sendTransaction when their async raises were updated with SignerError
Ongoing:
- Continue work on updating nim-ethers to support json-rpc breaking changes
- Continue work on supporting json-rpc breaking changes and pulling out utils/json to its own lib
- Integrate contract changes into nim-codex
- Look into removing waitFor in integration tests
- Review and clean up nim-ethers changes
- Try to figure out a cleaner way to handle exceptions instead of catching all CatchableErrors
- Start tweaking the nim-json api to normalize both serialize and deserialize pragmas, with modes: OptOut, OptIn, and Strict
- WIP on adding PATCH call for Availabilities
Research
2024 R&D Goals
1. Proving system and aggregation improvements (folding or lookups)
2. Aggregator/validator design
3. DHT improvements
4. Tokenomics and incentive design
5. Bandwidth incentives
6. Dynamic data (appendable data)
Completed:
- Frobenius endomorphism & pairing implementation
- Review the Solidity Groth16 verifier
Ongoing:
- DAS simulator improvements to cover more diffusion models
- Start DAS sample query mechanism design
- Proof recursion ideation