2023-10-25 Codex weekly

Codex Update Oct 2 - 23 §

We had a teamwide offsite in Crete from October 6th to the 12th, during which we were able to discuss and come up with concrete solutions around several important pieces of the project aspects such as the proving system and the contract start interactions, as well as plan out outstanding work for our upcoming testnet launch at the end of the year. The meeting was very productive and it helped align the team around the current and future outstanding work in order to hit our big end of year milestone.

Client §

Milestone: Block Merkelization §

• Merkelization concrete PR in review
  • https://github.com/codex-storage/nim-codex/pull/566

Milestone: Block Exchange protocol refinements and simulations §

• Some results were presented during the offsite
  • https://rpubs.com/giuliano_mega/codex-swarms

Milestone: Async Disc Access & Threading support §

• Work on IO threads support (not much progress due to offsite, will be picked up asap)
  • All related PRs are here https://github.com/codex-storage/nim-datastore/pulls
    • We now how a clear idea of how to implement and integrated it - https://github.com/codex-storage/nim-codex/pull/552

Milestone: Sampling and Storage Proofs §

• Work on storage proofs is ongoing under https://github.com/codex-storage/zk-research-artifacts/tree/master/storage_proofs
  • A circom implementation is being worked on in https://github.com/codex-storage/zk-research-artifacts/tree/master/storage_proofs/MVP/circuit
  • Reference/testing implementation to consume the circuits is done here https://github.com/codex-storage/zk-research-artifacts/tree/master/storage_proofs/MVP/reference
• Analysis of the sampling strategies are available here https://github.com/codex-storage/zk-research-artifacts/tree/master/sampling

Milestone: Client stability and debugging §

• Major effort to stabilize the Codex client through continuous automated testing
  • Work on better log analysis is being done here https://github.com/gmega/logtools
  • Running several load tests with ˜100 nodes using the dist testing framework https://github.com/codex-storage/cs-codex-dist-tests

Infra §

• Investigating and fixing issues related to Elasticsearch logshipping
  • Install Loki in Dist-Tests cluster
  • Fix Vector errors related to the Kubernetes logs shipping
  • Configure Vector to ship Dist/Continuous-Tests logs to Loki

Research §

Milestone: Publications §

• White paper ongoing - https://docs.google.com/document/d/1LCy23m90IHf32aUVhRT4r4772w1BfVcSLaJ0z9VTw9A/edit#heading=h.qs3bayckj5u4

DAS §

Milestone DAS §

• Our current engagement with the EF is coming to an end a summary of the work done has been collected here
  • https://hackmd.io/TYqF-wj2SIWwpS2F_PhWzg