Executive Summary
The month of September saw an agreement and solidification of the Waku roadmap which defines the process of launching the Waku Network as an independent piece of infrastructure the broader ecosystem can rely upon. Along with this, a revamp of the process in which work is labeled and tracked was performed which has an automated part and is generally more in line with the requests from the Insights team.
Work continues in the scaling and productionization efforts across the clients. Work previously done to enables PostgreSQL engine for WAKU-STORE in nwaku is being locally stress tested, further stress test using dev fleet is expected to be finalized next month. A Waku static sharding strategy is being integrated into Status to accommodate the first growth milestones of the re-release and adaptive sharding research and implementation is close behind.
Local simulation of RLN-RELAY has enabled the discovery of minor bugs which are fixed in the latest release of zerokit and nwaku and the study of the affect of RLN on performance.
Kurtosis as a platform was found to be insufficient for modeling a Waku network at the scale we wish, and the Vac team is pursuing an alternative strategy and writing up learnings from the boundaries were able to push.
Quality Assurance practices are scheduled throughout the next year to keep all client implementations up to a threshold of continuous quality.
The process to add native integration APIs to nwaku is underway such that the default native library moves from go-waku to nwaku.
Key Updates
Personnel
- addition of
- Aaron as Project Manager
- Sergei as Researcher
- Gabriel as nwaku Engineer
- Several Jobs Descriptions have been reviewed this month to be opened shortly:
- Growth Lead/Marketing strategist to drive Waku’s growth and liaise with Comms Hubs
- Business Development Lead, to further develop partnership with ecosystem projects
- Solution Engineer, to provide technical support to projects integrating Waku
- A core contributor to lead the Waku Chat SDK team has been secured, with start date in November.
Milestones
A lot of work has been put into coalescing and finalizing the development tracking process that is in line with the Insight Reporting requirements, and Aaron’s addition to the team this month as pushed it over the edge to completion. Much of this has gone into automating the weekly reporting process via GitHub labels and comments on issues.
For tracking Waku maintains these Milestones in the waku-org/pm repo. Within each milestone description, you’ll find the corresponding Epics. Every Epic is distinctly labeled, and this label is affixed to each issue associated with that particular Epic. The labels are managed by the labels.yaml file located in the waku-org/pm repo.
Given the expansive nature of Waku and its various repositories working towards the milestones, the labels established in the labels.yaml file are replicated across each respective waku-org repo. This structure allows for seamless navigation, starting from top-level milestones down to the most granular issues.
Waku is broken out into the following four Milestones, with Epics associated with them:
Waku Network Gen0Waku Network Can Support 1MM UsersQuality Assurace Processes in PlaceSupport Many Platforms
More details on the structure and progress of all Waku work can be tracked in their PM repository, specifically the milestone page. The following sections are highlighted updates on what happened this month.
Waku Network Gen0
The Waku Network RFC was created and published on Vac as RAW which details the ideas and architecture of the Waku Network. The next version of RLN was also published on Vac as RAW.
A benchmark of RLN was conducted and the results were discussed in a Logos Research Call presentation (See waku-org/research#23 for details). The tl;dr is copied here for convenience:
TLDR:
- Proof generation is constant-ish. 0.15 second for each proof
- Proof verification is constant-ish, 0.012 seconds. In a network with 10k nodes and D=6 this would add an overhead delay of 0.06 seconds.
- Gossipsub scoring drops connections from spammer peers, which acts as the punishment (instead of slashing). Validated in the simulation.
- RLN doesn’t have any impact on memory consumption.
Based on these two specification publications and other associated work, efforts have begun to launch the first dev-testnet in time for the DevConnect event in November 2023.
All launch critical work for autosharding has been done in terms of RFC and nwaku.
Waku Network Can Support 1MM Users
Significant work was completed on the PostgreSQL integration and setup within nwaku, which supports the data retention and retrieval of Waku archival nodes. The implementation is currently being stress-tested to ensure production performance metrics are met.
The efforts in simulating a Waku Network of 10k users within a single shard continues and is tracked within Vac DST Roadmap. The performance of Wakurtosis (Kurtosis backend) was found to be insufficient for our requirements for scaling simulations. The creation of a Kubernetes orchestration tool, written in Python, has begun construction. This tool is heavily architected to mimic what Codex has created. It was chosen to reproduce this tooling in Python in order to increase usability and ease of maintenance/contribution as C# is a less known language within the org. The reasoning for this development can be tracked in retrospective-rlog.
The effort to understand a “professional Waku node operator” has begun and initial notes can be tracked within this minutes doc.
A “static sharding” fleet was setup to test sharding and PostgreSQL by Waku team. Setup of a similar fleet dedicated to Status Communities is in progress (the first fleet may be used in the interim).
Quality Assurance Processes in Place
This milestone was created to ensure preparedness for the upcoming production client needs (specifically Waku Network Gen0 and Status Communities). A list of required processes in place was constructed and tasked out so that all implementations of Waku go through a standardized production release cycle.
This work is coordinating with the new Vac DST additions focused on testing rubrics for Logos Projects. This milestone is expected to be completed by Q1 2024. Work has started to use the js-waku CI as an integration test suite for nwaku and go-waku. This test suite can now easily be run for either client as part of their release process.
Support Many Platforms
This is a large milestone created last month that tracks Waku’s “integration landscape” and attempting to ensure any developer seamlessly is able to integrate Waku.
It has started to list the work required for completion but more detail is needed to be fleshed out on prioritization and estimated resource needs. Currently, it is slotted for completion by April 2024.
Much of the work this month was fleshing out the available REST APIs of nwaku.
A poll was created to query what language priority we should have was gone. They’ll be published on socials next month to boost engagement and feedback. This poll will assist with priorities for this milestone.
Perceived Changes in Project Risk
- Waku doing most integration of Waku into
status-goconsumes a lot of Go-related developer resources.- a list of needed work is tracked here
- There effort to convert
nwakuto the main native integration client requires a large effort in the implementations of C-bindings in Nim and has some unknowns associated with it. Furthermore this additional effort and uncertainty doesn’t directly contribute to the current critical path of development. - The first main application of the milestone reorganization within the project has made the milestones associated with it clear, thus allowing the Waku Network MVP target setup to be tracked well
Future Improvement Plans
Insight
The insight team plans to further evaluate the value the reporting process implemented by Waku as it pertains to use within the other projects under Logos. It is expected that next month it will be finalized and ready for review by other teams to see if they’d like to adopt it.
One side effect of the automated reporting process is that the associate issue labels are already compatible with our data lake ingestion that was initiated by the Status project. This will allow us to create more useful dashboards and monitoring that take into account accurate development activity.
Project
As the milestones continue to be fleshed out and detailed, the ability to show progress over time will improve.
Sources and Useful Links
Weekly Reports