2023-12-11 Waku Weekly

Waku Network Can Support 10K Users §

• achieved:
  • fixed issue of store fleet not connecting to bootstrap fleet due to enr shards mismatch https://github.com/status-im/infra-shards/issues/23
  • continuing team-internal dogfooding of a test community using static sharding https://github.com/waku-org/pm/issues/97. See dogfooding report
  • benchmarked various ways for large postgresql deployments: https://github.com/status-im/infra-status/issues/37
• next:
  • continue dogfooding of Status Desktop with Status staging fleet with test community (https://github.com/waku-org/pm/issues/97)
• risks:
  • Fleet Ownership doc defines fleet maintainer and owner. Status team yet to clarify who is the fleet owner for Status Communities.
  • QA by Status team to be planned on staging static sharding fleet; Waku team has done internal dogfooding (report). Any change to the staging static sharding fleet should then be tested by QA before being deployed to prod (e.g. # of Postgres instances). Status has committed to this testing on 28Nov call.
  • Status team expressed will to deploy static sharding prod fleet and use it for all users: This is not recommended until proper QA is done on stagning static sharding fleet as it could impact other Status app activities.
  • Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev (1, 2, 3). See status-go#4057 for remaining work. Mitigation by on-boarding Chat SDK team since November 2023 to drive effort.
  • Dependency on Vac/DST to conclude ~1k nodes simulations; lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.

Targeted dogfooding for Status Communities §

• achieved: fix peer manager to take into account the ENR seq to determine if a peer is new or not and fix incorrect number of connected peers per topic. Register lightpush protocol correctly in go-waku. Drop pubsub topic bridging. Use shards.test fleet as default (in status-go and desktop)
• next: continue dogfooding / fixing issues

Waku Network can Support 1 Million Users - 2023-11-30 §

• achieved:
  • Researched various ways of deploying shared postgresql instances for large deployments: https://github.com/status-im/infra-status/issues/37
• next:
  • Pending DST simulations of 10k nodes gossipsub network.
• risks:
  • Dependency on Vac/DST to run 10k nodes simulations. Tracked under vac:dst:eng-10ktool.
  • Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
  • Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
  • lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.

Waku Network Gen 0 - 2023-12-01 §

1.4: Sharded peer management and discovery §

[nwaku] feat: Peer management with shard as a dimension

• achieved: sharded peer management and store pruning PR merged

1.2: Autosharding for autoscaling §

[js-waku] feat: make autosharding default node behavior

• achieved: open PR to reintroduce (but deprecate) name sharding alongside auto/static sharding without breaking APIs
• next: update all examples to use autosharding
• blocker: need review on https://github.com/waku-org/js-waku/pull/1723

Support Many Platforms - 2024-04-30 §

REST API service node §

[docs.waku.org] REST API/ NodeJS

• achieved: added references for the REST API

Other Work §

Research §

[research] Onchain RLN tree+root: Proof Of Concept

• achieved: We present a proof of concept change in the RLN contract to store the whole membership tree on-chain + its Merkle root. This lowers sync time from several minutes to a few seconds, but at a cost of x10 the membership insertion cost. It also makes light clients lighter since proof verification becomes stateless (Merkle root can be accessed onchain, without having to sync the tree). We also present go-waku-light, to showcase the newly introduced features and how they are meant to be used.

Enhancements §

[nwaku] chore: avoid blocking the whole waku node when retention policy is being applied

• achieved: avoid blocking the whole waku node when the retention policy is being applied

Documentation §

[docs.waku.org] Encryption documentation

• achieved: push initial draft for symmetric, ECIES, message signing
• next: merge and deploy encryption docs #145

[docs.waku.org] Docs general improvement/incorporating feedback (2023)

• achieved: add RN warning, add certbot instructions, improve nwaku-compose guide

Chores §

[nwaku] Bump vendor dependencies for release 0.23.0

• achieved: bumped nim-dnsdisc dependencies
• next: bump nim-waku dependencies

Eco Dev §

• achieved: 51 projects submitted at EthIndia hackathon