Codex Update Mar 26th - Apr 1st
The Codex team continues to make progress with various initiatives to wrap up the demo for the Q1/Q2 public testnet release. An internal testnet has been running for the past few weeks and has been used to test the latest version of Codex and can be accessed using the Codex Testnet Starter documentation.
Ongoing and new lines of research and development will soon begin in preparation for the next version of Codex that will be used for the mainnet release.. Here are the updates from different team members and their ongoing work. Development is currently broken into three distinct teams:
- Client, Testing, and Infrastructure
- Marketplace
- Research
The different teams have actively moving on various fronts. The following are their team updates to various ongoing Epics.
Client, Testing and Infrastructure
Client
Completed
- Created task for NAT traversal and gathered various info for it https://github.com/codex-storage/nim-codex/issues/753
- Finished work on scheduling erasure coding on a thread pool https://github.com/codex-storage/nim-codex/pull/716
Ongoing
- Working on scheduling prover computation on a thread pool, draft PR in progress
- Addressing comments for large PRs
- https://github.com/codex-storage/nim-codex/pull/631
- https://github.com/codex-storage/nim-codex/pull/678
Testing
Completed
- Documented issue for running Codex integration tests on certain macOS configurations https://github.com/codex-storage/codex-contracts-eth/issues/95
Infrastructure
Completed
- None
Ongoing
- � Install Fluent Bit to get Testnet Kubernetes events
- � Adjust Vector config
- � Deploy ethereum web wallet for Testnet
- � Run Public RPC node on Testnet
- � Add more Codex Storage nodes
- � Use multi-stage builds for Rewarder #101
- � codex-testnet-starter updates
- �️ Configure Grafana dashboards for Testnet resources #154
- �️ Add Release workflow for Codex #749
Nim Tooling
Completed
- Add recursion limit for SAT (packaging) https://github.com/nim-lang/sat/pull/3
- Disable stacktrace on SAT solver to avoid stackoverflows in debug builds https://github.com/nim-lang/sat/pull/5#event-12262613353
- Wrote up draft for Nimble Caching RFC https://hackmd.io/@elcritch/rJ4VrExJR
Marketplace
Completed
- Revisit design of https://github.com/codex-storage/nim-codex/issues/467
- Review https://github.com/codex-storage/nim-codex/pull/692
- Review https://github.com/codex-storage/nim-codex/pull/678, many reviews and ongoing discussion
- Review https://github.com/codex-storage/nim-codex/pull/730
- Chore to remove no longer used compilation flag: https://github.com/codex-storage/nim-codex/pull/741
- Tried to understand why an integration was indeterminately failing in CI and locally, thought was very difficult to reproduce
- Merged https://github.com/codex-storage/nim-codex/pull/704, which had the integration test failure. Re-running the failed test in CI was enough to get it to merge, but indicates there is an issue with this test
- Rebased and merged MarketError PR: https://github.com/codex-storage/nim-codex/pull/670
- Updated pausing slot queue design PR: https://github.com/codex-storage/codex-research/pull/188/
- Re-reviewed feat[marketplace]: add slot queue pausing PR #752 in Codex and started review of Expiry per dataset PR #678
- Added protection branch rule for
nim-ethers
in Codex - Reviewed PRs #757, #752, #760, #69, and #70 in Codex
- Reviewed comments for the Availability Patch PR and Pausing Queue PR in Codex
Ongoing
- Discussed with team plans to improve integration tests: what they should cover, parallelism, and whether or not some could be replaced with unit tests. Also discussed some open issues and PR comments.
- Continued work on implementation of slot queue pausing. Most implementation in, need to add tests: https://github.com/codex-storage/nim-codex/tree/feat/marketplace/slot-queue-improvements
- Worked on updating a macro for Optionalize to add the serialize pragma annotation on the type. Tried to get the macro to read existing serialize annotations on the type and fields first but was taking too long, so moved on
- Discussions about release branches, proof circuit assets, and versioning
- Discussed
Optionalize()
macro andserialize
pragma - Discussed the parallelism of integration tests
- Progress on Persistent Availabilities implementation
Research
2024 R&D Goals
1. Proving system and aggregation improvements (folding or lookups)
2. Aggregator/validator design
3. DHT improvements
4. Tokenomics and incentive design
5. Bandwidth incentives
6. Dynamic data (appendable data)
7. Encryption
Completed
- None
Ongoing
- Trying to understand what can be salvaged from my binary circuit idea (by implementing and measuring stuff)
- Working on the Plonk notes