Read as Needed - Building WiSER, a Flash-Optimized Search Engine

Explore the development of WiSER, a flash-optimized search engine, in this 31-minute conference talk from FAST '20. Discover how the "read as needed" philosophy drives the design of this innovative system, which leverages high-performance SSDs to deliver improved throughput and reduced latency. Learn about key techniques such as optimized data layout, a novel two-way cost-aware Bloom filter, adaptive prefetching, and space-time trade-offs. Understand how WiSER outperforms state-of-the-art alternatives like Elasticsearch, achieving up to 2.7x increased query throughput and 16x reduced latency, even with limited main memory. Gain insights into the potential applications of the "read as needed" approach as storage device read performance continues to improve.