HN

Comments (61)

• MiroslavPokorny
  Why would any website want to put extra effort to "give" their content as markdown to AI and get nothing in return ?
• Menu

  k1m
  I agree with Roy Fielding on this:> It is a bad design trade-off to send a bunch of header fields on every request just to tell the server all of the possible variations of preference held by the user, particularly when there is a very small chance that any of those dimensions are applicable to the target resource. It has been a bad design trade-off ever since the very brief period in 1993-94 when folks didn't know which image format would be usable on all UAs and there was no CSS or javascript to allow for client-side adaptation.> ...The caching impact of proactive negotiation is far worse than the one extra round trip per site for reactive negotiation, and even that round-trip isn't necessary in formats that support client-side adaptation.On the caching impact, Simon Willison wrote:> ...you can’t deploy an application that uses content negotiation via the Accept header behind the Cloudflare CDN — for example serving JSON or HTML for the same URL depending on the incoming Accept header. If you do, Cloudflare may serve cached JSON to an HTML client or vice-versa.Note: I posted this in another comment with links to those two quotes which I couldn't copy easily now - will add later.
• Menu

  joshum97
  I feel like I am going insane. Who in their right mind would feed an LLM raw HTML in the first place??HTML is a markup language. User agents present it in a way that makes sense for the user—visually, or through assistive technology. Nothing about adding LLM “users” should change this—their user agent, the harness, should present HTML in a way they can natively understand, by translating it to Markdown.We are not going to rewrite the entire web because harness developers are too lazy or stupid to pull an HTML to markdown package from npm. If some sites want to do so anyway, good for them, in many cases I’d love to skip the CSS/JS and read the markdown directly (or better, nicely formatted). But don’t blame website authors for your harness wasting your tokens.
• Menu

  lekevicius
  I'll do that once any of the top 4 AI chatbots says they'll start making requests with this header. Before that it's just a neat idea with no adoption.I also think it's exceedingly unlikely that any of the top 4 chatbots would choose to load websites this way. Too many risks for a 0.01% adoption even years later.
• Menu

  collimarco
  What about clean, semantic HTML?It was already optimized for bots and search engines (which are bots) and it has been used for decades. Why we need to serve in markdown now?There are also many parts of the HTML, like navs, that are useful for bots and AI and may be removed in the markdown version.
• zzo38computer
  The web page does not load for me (I cannot reach the server), but the idea to serve Markdown according to the Accept header is possible, although it might be better to do that by changing the file name and adding a header (or a <meta> command) to indicate this possibility, so that there is a different file name in each case. Some already do this, allowing to change the file name for JSON or Markdown instead of HTML, and is helpful (there are also other common ways that you can change a URL to access it differently e.g. "blob" to "raw" in a git viewer, and I use these things sometimes). However, the reason should not be mainly for AI agents, even if it can be helpful for that purpose too.
• Menu

  kaangiray26
  hoping for this to get mainstream so that I can just view the pages without any ads, js and bloat
• Dwedit
  How about allowing browsers to actually display markdown?
• maurelius2
  Even without AI this would be a nice feature.
• LunicLynx
  Hello … AI company that wants easier access to data
• Menu

  arjie
  Everything is determined by cloudflare. If they supported Gemini on the same hostname I’d serve it everywhere. If they supported text/plain and text/html everywhere I’d serve that. But they don’t cache per content encoding so I just do what works for them.They’re far too useful for me.
• Menu

  Kuyawa
  Interesting...A cherry.jpg image/jpeg file is shown as an image by the browserA logo in svg like https://news.ycombinator.com/y18.svg tooSo browsers should show readme.md as text/markdown and clicking on a link inside that file should also redirect to another markdown file, instant wikiBrowsers should implement that asap and we will follow
• Menu

  a2ff6eeb0
  Hm, interesting avenue for prompt injection.
• brap
  Best case scenario, this ends up being abused in order to feed LLMs crap responses (or worse).
• monneyboi
  Yeah, this is the way forward. All the POW stuff is such a waste of energy.The open internet deserves a real solution instead of gatekeepers in the form of brightdata, firecrawl, cloudflare and the likes.
• nozzlegear
  No, I don't think I will. I publish things for people, not bots.
• Menu

  4lb0
  Is this an alternative or complements llms.txt?
• OutOfHere
  It's a good thought, but the problem with it is that it can never be trusted by the bot, except for a finite whitelisted set of trusted sites that are known to serve representative markdown that faithfully represents the HTML page, its content, and its links.
• anon
  undefined
• akomtu
  This will be used to serve SEO slop. LLMs are already used as search engines and businesses really want to outrank competition by any means possible.
• ErroneousBosh
  So can I use this to serve actively harmful content to the Cocaine Piracy Parrots?Maybe tarpit them into reading gigabytes of Markov Chain nonsense, really slowly?
• xgulfie
  Sounds like a great way to spread disinfo to LLMs, so I'm down
• zangxin22
  [flagged]
• wakelog
  [dead]
• postal6666
  [flagged]
• shahariaa
  [dead]