HN

Comments (49)

• Menu

  jitl
  This style of reactive programming is quite popular in JavaScript UI frameworks these days under the moniker “signals”, with a proposal for standardization here: https://github.com/tc39/proposal-signals#-javascript-signals...It’s used by frameworks Vue, SolidJS, Svelte, Ember, Angular, and there’s a few different implementations for React like Mobx and Jotai. There’s a few different algorithms for how to propagate changes and evaluate the DAG, I believe SolidJS2 uses a height-based algorithm similar to Incremental.I’ve been fooling around with an implementation that uses an Int32Array arena to allocate nodes and link them together with linked lists without paying O(dependency edges) GC load: https://github.com/justjake/dalien-signals/tree/dalien-signa...There are a few of these for Rust as well, Leptos is an example in UI frameworks, and Salsa is an example in general incremental computing, used in rust-analyzer.Another way to look at this sort of thing is as a build system with automatically tracked dependencies. One such build system is tup, which instruments build jobs to detect what files they read to establish dependency relationships. Interesting reading from the author: https://gittup.org/tup/build_system_rules_and_algorithms.pdf, see also the classic Build Systems à la Carte https://www.microsoft.com/en-us/research/wp-content/uploads/...
• Menu

  ronfriedhaber
  This is cool.As far as I can tell, incremental the library aims to solve the problem of partially hydrating a computation graph when source data is altered. This approach is similar to the one pursued by (well designed) build systems and is common in the FP world. [2] This has many use cases and is very cool.In addition, in the sphere of incremental computation, there exists Differential Dataflow, Timely Dataflow (adjacent), and DBSP. Systems like Feldera are built on DBSP. Materialize is lead by some DD guys.Personally, I am pursuing an orthogonal approach specifically for the problem of financial data and financial workloads, There exists huge, very important problems to solve! [1][1] https://modolap.com[2] Signals And Threads episode on the subject https://signalsandthreads.com/build-systems/
• Menu

  fadesibert
  Goldman took the same approach with instrument pricing ~30 years ago. I recall long discussions about "Node Purpling" in my ~13 year tenure there.Computer Science has evolved, and AFAICT this is not a graph approach, but things like differentiation are computationally expensive, and therefore you want to minimize the number of times you do it to as close to the theoretical minimum.Edit: Related HN discussion https://news.ycombinator.com/item?id=36006737
• Menu

  jimaway123
  Is anyone aware of a version of this focused on very speed sensitive, low-level incremental computation? Like perhaps a compiler that generated a kernel for doing an incremental computation on a static graph?
• djtango
  I was very curious about Dataflow programming years ago - I think a lot of people were coming at this problem from various angles. This specific library immediately reminded me of Javelin from Clojure [0][0]: https://github.com/hoplon/javelin
• osener
  If you find this interesting, also check out their UI library called Bonsai that built on top of Incremental: https://github.com/janestreet/bonsai/Libraries like React are pretty efficient with skipping work by using Virtual Dom, but constructing this vdom still takes time. Bonsai makes the vdom incremental and it is pretty fun to work with.I built a desktop UI library with it by targeting (now unmaintained) Revery. It is using a much older version of Bonsai however: https://github.com/ozanvos/bonsai_revery
• shortercode
  Oh cool I was working on something similar awhile back and couldn’t find much precedent. The project got retired as our use case went away. But I’d love to revisit it. It’s still on npm as “data-rambler”.Idea was a DSL that could be loaded into a JS runtime, then fed data streams. Your module would transform that data into various output streams which could then be fed to a separate reporting library. Powering dynamic reports based on a template.Our first iteration was already pretty powerful but I had some big plans to bring it into a more JS feeling syntax to reduce complexity.
• Menu

  RandomBK
  One thing I've never fully grokked is how this differs from an observable pattern where one can publish new values to inputs, propagate that through the computation, and push newly computed values to listeners.I guess there's probably optimizations around change detection and stopping the propagation if there's no change (though observables can do that as well). The stabilize command also makes things interesting as a way to batch changes together before recomputing (but again, doable with observables too).Is the delta primarily coming from introspection and automatically building the compute graph? Or is there something more fundamental that I'm missing?
• runtime_lens
  One thing i have always linked about Jane street projects is that they tend to package ideas that have existed in research or niche system into something developers can actually use. Even if you never adopt the library, the design docs are usually worth reading.
• lsuresh
  Over at Feldera, we focus on IVM for SQL, but incremental computing problems show up far and wide: UIs, spreadsheets, control planes, compilers and more.
• Menu

  evomassiny
  Can't you solve it using hash trees (or Merkle trees) ?You tag each computation nodes with a hash of its dependencies and some constant salt, that gives you an ID which identifies the results that the computation node would produce; before running it.You can then use those IDs to index the computations results in a cache; whenever you query a computation results, as long as you update the IDs of each leaf of the computation graph, you will only re-compute the nodes that need to be updated
• pgt
  Electric Clojure does incremental rendering that crosses the client/server boundary: http://electric.hyperfiddle.net/The closest thing to Electric (IMO) is SolidJS, but frontend only: https://www.solidjs.com/
• srean
  It might be worth reading this in concert withhttps://hn.algolia.com/?q=flow+based+programming+
• geokon
  Where is it actually explained how it works..?Usually these kinds of systems either don't scale dynamically or have caching issues. The first example, a spreadsheet, is "easy" because there are a fixed amount of cells to track. A GUI can be a lot harder (imagine sub windows and sub-sub windows dynamically popping up and tracking some redundant and some unique "computations". Entities can appear and then be removed at random). Though the wording carefully says "constructing views" so maybe it doesn't handle dynamism
• erichocean
  Parallel Self-Adjusting Computation[0] is a fun entry in this genre.[0] https://github.com/cmuparlay/psac
• iamwil
  For those interested in incremental systems, I recommend checking out DBSP. I thought it was pretty neat.
• raphinou
  I think websharper's Var are similar to this, and it is really great to develop dynamic web interfaces (in fsharp).
• geoHeil
  I maintain a similar library more focused on data engineering needs: https://docs.metaxy.io/ maybe it is useful for more people.
• xiaodai
  pardon my ignorance but is Ocaml performant enough? Why isn't something like this coded in say, C++?
• Menu

  Balinares
  Whenever I see something in OCaml I assume it's Jane Street and that ends up correct a surprising amount of the time.
• dh2022
  In C# a dependency graph that automatically updates only the affected dependencies can be implemented using events and/or functors and/or data binding.I do not understand what is the big deal with Increment. Is it more efficient because it is written in OCaml rather than C#?
• arthurbrown
  Another related but semantically distinct package with great developer ergonomics is the FRP library React -- https://erratique.ch/software/react/doc/React/index.htmlVery satisfying to use when you manage to find a problem that is suited to this type of approach.
• Menu

  mempko
  I have built something similar like this for my fund 7 years ago. We were doing parametric optimization on large computational graphs. I have never programmed Ocaml but my understanding is introspection is kind of a weak spot for the language. Curious language choice! I know Ocaml is fast, about 1-2x speed of C, on par with Java.
• Menu

  curtisblaine
  This reminds me of https://github.com/electric-sql/d2ts. Not sure they're comparable, but they seem to be aiming at the same problem.
• tangsoupgallery
  [flagged]
• Arshad-Talpur
  [dead]
• BedVibe_Studios
  [flagged]
• luciana1u
  [dead]
• jheriko
  [dead]
• reinitctxoffset
  Kobe!