2023-11-20 Waku weekly §

Waku Network Can Support 10K Users §

• achieved:
  • closed last PostgreSQL issue for Store scalability
  • confirmed no unknown blockers from Waku’s side to continue dogfooding in conversation with Status Communities
  • started team-internal dogfooding of a test community using static sharding
  • started fleet ownership handover process: published guidelines/list of responsibilities - https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741
• next:
  • continue dogfooding of Status Desktop with Status staging fleet with test community
  • training session to conclude fleet ownership handover: https://www.notion.so/Fleet-Ownership-7532aad8896d46599abac3c274189741
• risks:
  • Dependency on Vac/DST to conclude ~1k nodes simulations.
  • Implementation of static sharding in Status Communities and design decisions mostly driven by go-waku developer, with minimal input from Status dev (1, 2, 3). See status-go#4057 for remaining work. Mitigation by on-boarding Chat SDK lead on 6 Nov to drive effort.
  • lack of confidence in simulation results: results so far exhibits various artifacts and anomalies seemingly related to tooling limitations. It is therefore difficult to draw certain conclusions re Waku scalability.
  • lack of clarity in terms of Status fleet ownership, monitoring and maintenance, which is an integral part of the solution.

Targeted dogfooding for Status Communities §

• achieved: logout / login freeze, fix request on correct pubsub topic, and add missing shard information on community invite
• next: dogfooding

Waku Network can Support 1 Million Users - 2023-11-30 §

• achieved:
  • Closed last Postgresql issue for basic Store scalability. See 10k milestone.
  • Assisted DST in setting up initial tests with the ~1K tool. Currently still fine-tuning parameters, ensuring results are consistent, etc. for smaller configurations.
• risks:
  • Dependency on Vac/DST to run 10k nodes simulations. Tracked under vac:dst:eng-10ktool.
  • Wakutorsis tool is being dropped, meaning new tooling needs to be developed for 10k nodes simulations. It is currently uncertain whether such tool can be developed.
  • Large scale simulations done by Vac/DST only covered nwaku relay. go-waku, status-go simulations are not planned short term (theoretical review of Status Communities messages is), nor are simulations including request-response protocols such as store and filter.
  • lack of real world feedback/dogfooding: the complete static sharding solution involves significant changes to the Waku protocol and tech stack. Although each element is unit tested, dogfooding may hit corner cases in the integrated solution that cannot be foreseen/recreated in lab conditions.

Waku Network Gen 0 - 2023-12-01 §

• achieved:
  • Internal dogfooding of proto-network continues.
  • Significant progress of autosharding in js-waku: autosharding function implemented, work to integrate in protocols started.
• risks:
  • Usage of RLN in js-waku and dependency on a (centralized?) Web3Provider remains unclear as one needs to know the merkle tree state (on chain) to generate proofs.
  • We are progressively moving a nwaku engineer to a solution engineer role we need to backfill the role.
  • js-waku team is juggling between dev ex and gen 0 with only 2 engineers (3rd one just joined) so delivery in this client is likely to lag behind other clients.
  • Uncertainty as to how RLN membership mechanism would hinder application adoption, if memberships need to be distributed or obtained by registration, if staking is necessary to prevent abuse, etc.

1.4: Sharded peer management and discovery §

[nwaku] chore: improve cluster id, shards, topics flow

• achieved: Various tests updates and fixes.
• next: Figure out why CI passes locally only.

1.2: Autosharding for autoscaling §

[nwaku] chore: allow fetching cached messages by content or pubsub topic

• achieved: failed refactor of message cache
• next: a better and simpler message cache

[js-waku] feat: Autosharding API for req-resp protocols

• achieved: derive pubsub topic from content topic in encoders/decoders when autosharding is specified
• next: node config should specify static sharding or autosharding. implement autosharded topics in all req-resp protocols

Support Many Platforms - 2024-04-30 §

Presentation Readiness §

[js-waku] feat: node connection state

• achieved: modified ConnectionManager to track connection state. exposed through read function and new event
• next: resolve any remaining feedback

Other Work §

Enhancements §

[nwaku] chore(REST): adding to /admin/v1/peers response lightpush and filter v2 peer info

• achieved: implemented, tested and raised PR
• next: get PR reviewed, implement feedback and merge

[nwaku] chore: allow text/plain contentType for rest request’s body types

• achieved: Better support for js-waku using RestApi: Allow text/plain contentType
• next: Bandwidth management preparation

[nwaku] chore: decouple listen and announced addresses

• achieved: got the PR reviewed, implemented feedback and merged

Ecosystem Development §

• achieved:
  • Presented two talks last week. Logos + Waku at Web3Privacy meetup and RLN at DCxPrague.
  • EthGlobal Istanbul - 20 projects submitted and contributed with production and mentorship
  • secured talks and presented at Libp2p day and EthGlobal
  • represented Waku in pragma booth
  • Helped organise Status-wide meetup during DevConnect
• next:
  • publish tic-tac-toe blog post
  • migrate examples from personal repository to examples repository and take responsibility of examples repo
  • edit and send video tutorials for review for comms
  • write event proposal for co-hosting LibP2P day in EthIndia along with Protocol Labs during the Waku hacker house
  • EthIndia production works (finding venue, vendors for swag and booth)
  • process and deliver devconnect feedback
  • help draft devconnect winner promotions
  • onboard the devconnect winners to Waku discord to Hackathon-winners channel
  • prepare event report of Devconnect with help from Lou
  • prepare KPI document for Eth India with help from Lou