HN

Comments (40)

• Menu

  red_admiral
  This is what you get when you let your ORM loose on the database without understanding JOINs. Especially, the bit where something like 'book.author.name' that looks like a simple field dereference actually is a method call on an ORM proxy object (book), via python's __getattr__ or similar, that fires off a new query if the data you want is not loaded yet.Some ORMs let you specify the extent of the data that you want, like Hibernate has its own Hibernate Query Language.At some point you are better off just writing SQL yourself, though. Even without join problems, if you ask an ORM to get the person with user id 123 and all you want is their name, the ORM cannot know that unless to tell it, and so you end up with a 'SELECT *' type query.
• WilcoKruijer
  I really believe that every engineer writing queries (even SQL) should read the FoundationDB data modeling guide [0]. It really gives an appreciation of what smart choice of primary key can do to query efficiency. With some de-normalization, joins aren’t even needed for performance.Postgres has supported query pipelining for a long time. In my opinion, most queries should be written in such a way that sequential queries don’t have any data dependencies on the previous query at all. This speeds up applications by huge amounts.[0] https://apple.github.io/foundationdb/data-modeling.html
• Menu

  quibono
  Nice, and I understand why using `getAuthorNames` solves the N+1 here.But... isn't this solving the problem by removing most of what makes it an issue in the first place? I imagine most people use ORMs for the SQL <-> native class data sync capability. And this assumes one would run the Acadia query instead.FWIW I'm not trying to be negative, it's just my general impression is that these N+1 usually occur because people _want_ direct object access and _want_ to write loops, and _want_ to access fields and have the underlying SQL be sorted by the ORM.
• wood_spirit
  Of course mainstream ORMs leave a lot of perf on the table. For example, I once patched the ORM in a struggling php web app that i had to help. I started as a logger profiler thingy but then had the crazy idea of being a trace optimiser. By recognising the call sites from previous visits I could spot the 1+N and select * etc and actually transcode that into better sql in the next run etc. Shockingly it made a massive difference and I was surprised that normal ORMs aren’t doing that kind of thing.
• Menu

  kstrauser
  Side note: I strongly prefer referring to this as the "1+N problem" as the author did here. I didn't understand what people were grousing about when they talked about "N+1".N+1: You're already doing N queries. Is adding 1 more that big of a deal?1+N: This should have been 1 query, but somehow you blew it up into that one plus N more.I'd seen that query antipattern plenty of times and knew what it was bad, but didn't realize that's what people meant by "N+1", which I thought must mean something different.
• Menu

  vilterp
  > [Datalog] is a subset of Prolog that lacks recursionDatalog does allow for recursion — a common example is graph reachability:reachable(a, b) :- edge(a, b). reachable(a, c) :- edge(a, b), reachable(b, c).(Evan mentioned implementing kCFA, which would require recursion like this...)'Base datalog' guarantees termination by requiring all input relations to be finite. Notably this means that it doesn't have numerical operations like addition or multiplication, since `plus(a, b)` or `times(a, b)` would be infinite relations.More practical Datalog engines like Souffle (https://souffle-lang.github.io/) have numerical operations but don't guarantee termination.Recursive queries are not needed by most applications, but maybe Acadia could allow them (compiling to recursive CTEs) by proving that recursion only goes through finite relations.
• mrkeen
  Compare with the prior art of N+1 queries of 2014:https://github.com/facebook/Haxl/blob/main/example/sql/readm...
• Menu

  koliber
  This is largely a solved problem. New generations of programmers are simply rediscovering it.The solution:- be aware of it- add DB query monitoring via your favorite APM tool- review the APM tool regularly- when you see an N+1 issue apply one of the normal solutions.Follow this and N+1 query issues will disappear soon enough.If you can’t do this it means you chose an immature framework or tech stack and my advice is to consider starting from scratch. Otherwise you will need to re-live the mistakes many people have already solved before you which feels adventurous but is painful and dumb.
• Menu

  jbverschoor
  Wasn't this 'solved' by Hibernate ages ago?
• anon
  undefined
• hmnxr1e
  Non-identity principle.A=A
• gigatexal
  just write sql smhit's so easy to get proper queries and then the mapping from a list of tuples to your object is easy