HN

Comments (41)

• carlsverre
  (I used to work at SingleStore, and now work at Antithesis)SingleStore (f.k.a. MemSQL) used lock-free skiplists extensively as the backing storage of their rowstore tables and indexes. Adam Prout (ex CTO) wrote about it here: https://www.singlestore.com/blog/what-is-skiplist-why-skipli...When SingleStore added a Columnar storage option (LSM tree), L0 was simply a rowstore table. Since rowstore was already a highly optimized, durable, and large-scale storage engine, it allowed L0 to absorb a highly concurrent transactional write workload. This capability was a key part of SingleStore's ability to handle HTAP workloads. If you want to learn more, take a look at this paper which documents the entire system end-to-end: https://dl.acm.org/doi/10.1145/3514221.3526055
• Menu

  ozgrakkurt
  Some more links that are inside the article:- More info about skiplists: https://arxiv.org/pdf/2403.04582- Performance comparison with B-tree ?: https://db.cs.cmu.edu/papers/2018/mod342-wangA.pdf- Other blog from Anthithesis about writing their own db: https://antithesis.com/blog/2025/testing_pangolin/Also I find it a bit hard to understand the performance outcome of this setup.I know formats like parquet and databases like ClickHouse work better when duplicating data instead of doing joins. I guess BigQuery is similar.The article is great but would be also interesting to learn how performance actually worked out with this.
• Menu

  cremer
  Redis sorted sets are probably the most widely deployed example. Redis uses a skiplist for range queries and ordered iteration paired with a hash table for O(1) lookups. Together they cover the full API at the right complexity for each operationSkiplists also win over balanced BSTs when it comes to concurrent access. Lock-free implementations are much simplier to reason about and get right. ConcurrentSkipListMap has been in the standard library since Java 6 for exactly this reason and it holds up well under high contention
• Menu

  josephg
  For this problem, I’d consider a different approach. You have a fuzzer, and based on some seed it’s generating lots of records. You then need to query a specific record (or set of records) based on the leaf.I’d just store a table of records with the leaf, associated with the seed. A good fuzzer is entirely deterministic. So you should be able to regenerate the entire run from simply knowing the seed. Just store a table of {leaf, seed}. Then gather all the seeds which generated the leaf you’re interested in and rerun the fuzzer for those seeds at query time to figure out what choices were made.
• Menu

  bob1029
  On practical machines they aren't good for much. To access a value in a skip list you have to dereference way more pointers than in a b+ tree. On paper they're about the same, but in practice the binary tree will tend to outperform. You get way more work done per IO operation.
• winwang
  Only somewhat related but there is supposedly a SIMD/GPU-friendly skiplist algo written about here: https://csaws.cs.technion.ac.il/~erez/Papers/GPUSkiplist.pdf
• Menu

  ahartmetz
  Skiplists have some nice properties - the code is fairly short and easy to understand, for one. Qt's QMap used to be skip list based, here's the rationale given for it: https://doc.qt.io/archives/qq/qq19-containers.html#associati...
• Menu

  teiferer
  In the age of agentic programming and the ever increasing pressure to ship faster, I'm afraid this kind of knowledge will become more and more fringe, even moreso than it is today. Who has the time to think through the intricacies of parallel data structures? Clearly we'll just throw more hardware at problems, write yet another service/api/http endpoint and move on to the next hype. The LLM figures out the algorithms and we soon lose the skills to develop new ones. And tell each other the scifi BS myth that "AI" will invent new data structures in the future so we don't even beed humans in the loop.
• mrjn
  skiplists form the basis of in-memory tables used by LSM trees, which are themselves the basis of most modern DBs (written post 2005).
• medbar
  Skiplist operations are local for the most part, which makes it easier to write thread-safe code for than b-trees in practice. Anecdotally, they were a nice implementation problem for my Java class in uni. But I liked working with b-lists more.Skip trees/graphs sound interesting, but I can't think of any use case for them off the top of my head.
• tooltower
  In my personal projects, I've used it to insert/delete transactions in a ledger. I wanted to be able to update/query the account balance fast. Like the article says, "fold operations".
• shawn_w
  Random access with similar performance to a balanced binary tree, and ordered iteration as simple as a linked list. It's a nice combination. (Of course, so is a binary search of a sorted array, which I lean more towards these days unless doing a lot of random insertions and deletions throughout the life of the mapping).
• torginus
  >What are skiplists good forIn practice, I have found out, nothing much. Their appeal comes from being simpler to implement than self-balancing trees, while claiming to offer the same performance.But they completely lack a mechanism for rebalancing, and are incredibly pointer heavy (in this implementation at least), and inserts/deletes can involve an ungodly amount of pointer patching.While I think there are some append-heavy access patterns where it can come up on top, I have found that the gap between using a BST, a hashtable, or just putting stuff in an array and just sorting it when needed is very small.
• Menu

  aaa_aaa
  Almost nothing. My friend and I used it once (in a rather obscure problem). Then used simple lists with some tricks with better performance because of the locality etc.
• Menu

  torben-friis
  Could someone provide intuitive understanding for why the "express lanes" in a skip list are created probabilistically?My first instinctive idea would be that there is an optimal distance, maybe based on absolute distance or by function of list size or frequency of access or whatever. Leaving the promotion to randomness is counter intuitive to me.
• Menu

  fnordpiglet
  A major global bank operated all trading, especially the complex stuff, off of a globally replicated skip list.
• Menu

  locknitpicker
  FTA:> Skiplists to the rescue! Or rather, a weird thing we invented called a “skiptree”…I can't help but wonder. The article makes no mention of b-trees if any kind. To me, this sounded like the obvious first step.If their main requirement was to do sequential access to load data, and their problem was how to speed up tree traversal on an ad-hoc tree data structure that was too deep, then I wonder if their problem was simply having tree nodes with too few children. A B+ tree specifically sounds to be right on the nose for the task.
• esafak
  What they really wanted was an HTAP. https://en.wikipedia.org/wiki/Hybrid_transactional/analytica...
• m00dy
  It was really cool to mention its name during tech interviews but not anymore I guess.
• hpcgroup
  [dead]
• linzhangrun
  [dead]
• feverzsj
  If you need a graph db, use a graph db.
• Menu

  jimmypk
  The article's actual thesis is subtle — it's not "skiplists are faster" but "a totally naive implementation has adequate performance." That property was precisely why the skiptree-in-SQL approach worked: you needed a data structure where a half-implemented version, running on BigQuery's execution engine, would still satisfy the performance contract. A B+ tree has no such property — a naive B+ tree in SQL would be strictly worse than using recursive CTEs, because you can't maintain balance across JOINs.This is also why skiplists are attractive in research and in early-stage implementations: the randomized promotion converts write complexity (deterministic rebalancing) into write simplicity (probabilistic promotion). You get a weaker worst-case guarantee, but in practice — especially in distributed systems where latency variance is already high — losing the rebalancing phase is often a good trade. The Redis case is the same dynamic: they didn't choose skiplists over red-black trees for raw speed, they chose them because augmenting a skiplist with custom metadata (like the "span" antirez mentions) is local and composable in a way that augmenting a balanced BST is not.