2025-09-15 Codex weekly

Week 37 August 2025 §

FURPs for replacing Bittorrent in status-go §

FURPs requirements have been published for the prototype of Codex replacing Bittorrent in status-go for the Status Community History service.

Replace bittorrent in status-go §

1. Enable hole punching with circuit relay v2

   • achieved:
     • Investigated possible solutions for dynamic multicodec/multihash formats in nim-libp2p
     • Investigated (default) limits for circuit-relays. Also in other projects utilizing libp2p (IPFS).
     • Stumbled into the fact that we run on branched-off version of nim-libp2p. Explored possible solutions and decided to simply stay on the branch until either “dynamic multicodec/multihash formats” (#904) is implemented or we sunset the Poseidon2 hash function that is the reason for the branch off. Till then we plan to bring updates from master to the branch when needed.
   • next:
     • Continue with initial basic implementation of the libp2p NAT traversal components #1319
   • blockers:

2. BlockExchange Performance and reliability

   • achieved:
     • Analyzed BlockExchange architecture and identified performance bottlenecks in streaming downloads
     • Investigated bulk prefetch interference with individual block requests through systematic testing
     • Investigated peer coordination timing issues causing duplicate block requests
   • next:
     • Continue systematic investigation of peer discovery and block presence coordination
     • Research approaches to optimize individual block request latency
   • blockers:

3. Identify Swarm Participation Limits (DHT Scaling)

   • achieved:
     • Researched provider record liveliness and its affect in the re-advertise rate in our use-case.
     • Researched provider churn in peer-to-peer networks since it plays a significant role in the query rate in our setting.
     • Defined expected ranges for the core variables used in calculating per-provider bandwidth based on prior research.
     • Calculate bandwidth requirements for core operations across different network sizes, data sets, and churn rates.
     • Produced figures and tables summarizing these results.
     • Finalised the research document in a PR.
   • next:
     • Review and refine the research document.
     • Explore additional scaling options and possible optimizations.
   • blockers:

4. Implement a Codex library so that clients can use Codex directly from their apps

   • achieved:
     • First Go wrapper that starts and stops a Codex node.
     • Added Codex version and revision API.
     • Provided a better API with ACK and OK differentiation.
     • Fixed an async issue from the Nim library template.
   • next:
     • Add more Codex API features to the wrapper (debug, spr, etc.).
   • blockers:

5. Integration with status-go

   • achieved
     • Got better grip on the testing environment
       • managed to run unit tests which is where the core history archive functionality is tested, learning GO testing tooling,
       • functional tests - more complex and less stable. Now under control. Possibly not really crucial for the work to be done at this stage, but it is good to know the system is behaving predictability. Made some fine-grained scripts to run selected tests, all tests without coverage, building status-go docker image, and more robust cleanup,
       • improved reliability of functional test - selective running, waiting for docker services to be healthy before running the tests in order to avoid misleading re-runs and difficulties in cleanup,
       • got some basics of nix and flake environment which is used by status-go in CI. Decided not to use them during local development though (adds lots of complexity, and feels like it makes the machine dirty).
       • documented how to setup dev/test environment (still wrapping up some small bits, when ready the docs will be added via codex-docs-obsidian/pull/19 and the relevant docs will be exposed via HackMd, see e.g.: Running functional tests in development,
     • Draft PR to for the status-go-codex integration created (now only containing the above mentioned testing scripts modifications): status-go/pull/6930
   • next:
     • working on the actual integration of Codex with status-go (using Codex API calls)
   • blockers:

Research future work §

1. Altruistic Mode
   • Document: The first version of our simple model for DHT traffic is complete. This should allow us to keep discussions on whether or not the “DHT will scale” to support a proposed mechanism more grounded.
   • Document: Completed churn section for Assumptions for Durability under Altruism. This tries to convey the idea that durability under altruism is nuanced, and put together the factors that might affect it.
   • Document: Internal critique for why “Low Durability” is a bad name for altruistic Codex, and why it is also a symptom of a lack of proper problem understanding.
   • Document: Running list of known problems for altruistic Codex; we will be populating this every week.

Misc §

1. Testnet Decommission
   • achieved:
     • exported testnet discord bot logs
     • took testnet discord bot offline
   • next:
     • export record of testnet blockchain
     • take testnet blockchain offline