2024-01-29 Codex Weekly

Codex Update Jan 22nd - Jan 29th §

The Codex team continues to make progress with various initiatives to wrap up the demo for the Q1/Q2 public testnet release. An internal testnet has been running for the past few weeks and has been used to test the latest version of Codex and can be accessed using the Codex Testnet Starter documentation.

Ongoing and new lines of research and development will soon begin in preparation for the next version of Codex that will be used for the mainnet release.. Here are the updates from different team members and their ongoing work.

Development is currently broken into three distinct teams:

1. Client, Testing, and Infrastructure
2. Marketplace
3. Research

The different teams have actively moving on various fronts. The following are their team updates to various ongoing Epics.

Client §

Epic: Nim Improvements §

• Completed:
  • Filed Atlas issue to restore handling forked repos
  • Filed Atlas issue for handling “vendoring” style setups
• Ongoing:
  • Review Atlas graph updates and contribute

Epic: Wiring the Proving System §

• Completed:
• Ongoing:
  • Dataset expiry
  • Update DataSampler to match updated SlotBuilder
  • Investigate: way to run codex tests through valgrind or similar tool
  • Integrate DataSampler in Marketplace callbacks

Epic: Improve Client Stability §

• Completed:
• Ongoing:
  • Contibue rebasing of unmerged Chronos V4 PR and build improvements PR + fixes to initial idea on builds
  • Continue work on asynctest branch
  • Bug-fixing to track down the “file sizes differ” bug
  • Other bugs to be fixed

Marketplace §

Epic: End-to-end Testing §

• Completed:
  • Fix tests and replcae GPL licensed ZK verifier w/ MIT version that take JSON output from Circom in contract
  • Add formatting check to CI in codex-contracts-eth repo
  • Rebase Marketplace integration test refactor with erasure coding changes
• Ongoing:
  • Persistent Availabilities
    • Design Plan
  • Add duration to Codex frontend
  • Added a calendar for the expiry of creating ROSC’s
  • Work to add ceremony files to Codex
    • Ceremony file distribution
    • Add Ceremony file hash to Marketplace’s Contract configuration
    • Add support to retrieve Ceremony file hash from Contract’s configuration
    • Add support for Ceremony file hash download from S3
  • Continue work on updating nim-ethers to support JSON-RPC breaking changes and pulling out utils/json to its own library

Infra §

• Completed:
  • Deploy a site to store circuit ceremony files
  • Automate Codex ceremony files upload to Cloudflare R2 using GitHub Actions
• Ongoing:
  • Create Codex Helm chart
  • Deploy Codex Storage nodes in Testnet cluster

Research §

2024 R&D Goals
1. Proving system and aggregation improvements (folding or lookups)
2. Aggregator/validator design
3. DHT improvements
4. Tokenomics and incentive design
5. Bandwidth incentives
6. Dynamic data (appendable data)

• Completed:
  • Implement G2 curves (so most building blocks are now in place in my algebra backend to be able to experiment
  • Review work from Hashcloak collaboration for zk backend benchmarking test suite
• Ongoing:
  • New proof system design proposal