This requests dataset was created using a pool of Chrome headless browsers to visit home pages of the top 500 domains , as well as up to 3 pages of each domain and collecting all the network requests seen . The dataset was shuffled in such a way that the different pages were visited in a random order, but requests seen on each page were replayed as they were recorded initially.
The script to create the dataset is also available: create_dataset.js and shuffle_dataset.js was used to shuffle the requests to produce the final data.
It results from this observation that content-blockers will perform better on average if they can efficiently decide which requests to not block.
It should be noted at this point that a larger proportion of requests would be blocked by enabling extra filters lists such as EasyPrivacy.
Another use-case allowed by such capability is to perform the parsing of the lists on the backend and ship the serialized form of the content-blocker to clients directly, which removes the cost of initialization completely.
The reason is the following: the internal representation is already mostly stored in a compact form ; this means that serialization only consists in adding a small amount of metadata along-side the already available arrays and deserialization is essentially instantaneous since it's enough to create some typed array views on top of the serialized buffer . This also explains the very low memory consumption: after initialization, the memory usage is only slightly higher than the size of the serialized form.
Moreover, the efficiency of content-blockers is continuously improving, either thanks to more innovative approaches or using technologies like WebAssembly to reach native performance.
Original article