HN

Comments (80)

• Menu

  roywiggins
  > Elastic has been working on this gap. The more recent ES|QL introduces a similar feature called lookup joins, and Elastic SQL provides a more familiar syntax (with no joins). But these are still bound by Lucene’s underlying index model. On top of that, developers now face a confusing sprawl of overlapping query syntaxes (currently: Query DSL, ES|QL, SQL, EQL, KQL), each suited to different use cases, and with different strengths and weaknesses.I suppose we need a new rule, "Any sufficiently successful data store eventually sprouts at least one ad hoc, informally-specified, inconsistency-ridden, slow implementation of half of a relational database"
• Menu

  speedgoose
  Accenture managed to build a data platform for my company with Elasticsearch as the primary database. I raised concerns early during the process but their software architect told me they never had any issues. I assume he didn’t lie. I was only an user so I didn’t fight and decided to not make my work rely on their work.
• Menu

  PedroBatista
  I really never understood how people could store very important information in ES like it was a database.Even if they don't understand what ES is and what a "normal" database is, I'm sure some of those people run into issues where their "db" got either corrupted of lost data even when testing and building their system around it. This is and was general knowledge at the time, it was no secret that from time to time things got corrupted and indexes needed to be rebuilt.Doesn't happen all the time, but way greater than zero times and it's understandable because Lucene is not a DB engine or "DB grade" storage engine, they had other more important things to solve in their domain.So when I read stories of data loss and things going South, I don't have sympathy for anyone involved other than the unsuspecting final clients. These people knew or more or less knew and choose to ignore and be lazy.
• Menu

  lvspiff
  Everything is a database if you believe hard enoughFeel like the christmas story kid -->simplicity, and world-class performance, get started with XXXXXXXX.A crummy commercial?
• Menu

  jfengel
  No, of course not. But the question is, do you need a database?A database is a big proposition: transactions, indexes, query processing, replication, distribution, etc. A fair number of use cases are just "Take this data and give it back to me when I ask for it".ES (or any other not-a-database) might not be a full-bore DBMS. But it might be what you need.
• Menu

  vedhant
  Ofcourse it is not meant as a primary database. What baffles me is that people use it as log storage. As an application scales, storage and querying logs become the bottleneck if elasticsearch is used. I was dealing with a system that could afford only 1 week of log retention!
• Menu

  cluckindan
  ”That means a recently acknowledged write may not show up until the next refresh.”Which is why you supply the parameter refresh: ”wait_for” in your writes. This forces a refresh and waits for it to happen before completing the request.”schema migrations require moving the entire system of record into a new structure, under load, with no safety net”Use index aliases. Create new index using the new mapping, make a reindex request from old index to new one. When it finishes, change the alias to point to the new index.The other criticisms are more valid, but not entirely: for example, no database ”just works” without carefully tuning the memory-related configuration for your workload, schema and data.
• wwarner
  These drawbacks are all true, but sometimes storing directly to elastic is still the best way forward.
• jamesgresql
  I know it sounds obvious, but some people are pretty determined to us it that way!
• CodeCompost
  Yes but is it webscale?(Obviously I'm referring to a famous YouTube video on the subject)
• Menu

  toenail
  I think elastic always clearly documented to expect "eventual consistency", they never claimed to be a "database" in the sense that tfa defines.
• Menu

  unethical_ban
  I work in infosec and several popular platforms use elasticsearch for log storage and analysis.I would never. Ever. Bet my savings on ES being stable enough to always be online to take in data, or predictable in retaining the data it took in.It feels very best-effort and as a consultant, I recommend orgs use some other system for retaining their logs, even a raw filesystem with rolling zips, before relying on ES unless you have a dedicated team constantly monitoring it.
• Menu

  throw_m239339
  It has an index? It has data that can be queried with indexes? it is a database. PERIOD. Let's not turn the word database into a buzzword.It should obviously NOT be a "main" database but part of an ETL pipeline for search purposes for instance.
• gmuslera
  ... for a particular, opinionated definition of what a database should be.
• aaroninsf
  We use ES like a DB, but, not with SQL; and most importantly, it's not the source of truth/primary store. It's operational truth and best-effort.
• stefanon
  Yep!
• trgn
  it now has dedicated index types for logs and metrics with all kinds of sugar and tweaks in default behavior, they should introduce a new one called "database" that's acid.
• Menu

  this_user
  I mean, it is called "ElasticSEARCH", not "Elasticdatabase".
• alittletooraph2
  [dead]