HN

Need help?
<- Back
Web Search API

Web Search API

tosh

Comments (144)

  • simonw
    My number one question about search APIs is always if they allow you to store and resyndicate results you get from them.If I'm running an agent system but I'm not allowed to store the responses - or provide a "share transcript" button - that's a pretty significant limitation.The answer to that question is inevitably buried deep in the terms. Here's the relevant section I found for Ceramic, in their list of things you can't do:> (n) collect, aggregate, store, or compile Output, including search results, relevance scores, or rankings, for the purpose of creating or contributing to any database, dataset, index, or corpus, whether or not such database, dataset, index, or corpus is used for a purpose that competes with Ceramic; (o) resell, syndicate, or otherwise make Output available to any third party on a standalone basis or as a separately accessible component of another product or service; provided that you may display Output to your authorized end users within your own application so long as such Output is integrated into your application's functionality, is incident to the end user’s real-time query, and is not independently accessible, extractable, or downloadable by end users or third parties; or (p) retain, cache, or store Output beyond what is reasonably necessary to display such Output to your authorized end users in the ordinary and real-time course of use, unless expressly permitted in an applicable Order Form.https://www.ceramic.ai/terms-of-serviceAm I alone in caring about this?
  • iphonecorridor
    For those developers out there, the best is still Gemini Flash Lite 2.5 believe it or not. It gives you 1000 google searches per day for free. Compare to Flash Lite 3.x which is 5k PER MONTH and then a few pennies PER SEARCH. Nuts. Didn’t realize search was so expensive.Perhaps realizing all of this, Google hasn’t yet deprecated 2.5, bit limits access to it to “those who have used it before.”It’s really really good for low cost search!
  • binarymax
    Why not use those providers directly? Does Cloudflare need to be in the middle of everything?
  • qznc
    My coding agent uses the hister cli, i.e. a local index. That often requires me to seed it manually as a downside. The upside is that it caches website contents via browser plugin, which is a nice workaround for bot blocking.Thanks asciimoo for https://github.com/asciimoo/hister
  • jasonjmcghee
    > All three support Zero Data Retention for requests made through CloudflareAnd then on the providers page: Property Value provider exa Zero Data Retention No
  • karmakaze
    I was just looking into these as DeepSeek Harness w/ Qwen3.8-27B relies heavily on search. I was going to go with Serper.dev[0] $1 per 1000 (or lower in quantity).The providers[1] behind this Web Search API have very different rates: Ceramic.ai: $0.25 per 1,000 requests Linkup: $5.00 per 1,000 requests Exa: $7.00 per 1,000 requests [0] https://serper.dev/[1] https://developers.cloudflare.com/web-search/providers/
  • duncangh
    Cloudflare has been shipping more than FedEx lately. Would love to learn more about how they are going about this from a strategy, planning and execution standpoint.
  • innocent_name
    Of course Firecrawl isn't "Verified bot". Their customers were responsible for 95% of my traffic bill overcharge.
  • sreekanth850
    Create bot detection and bot protection, then sell crawlers. Is this the peak of hypocrisy?
  • ByeByeSpace
    I run a small side project on Cloudflare Workers, so having search available right from a Worker without adding another vendor is appealing. Curious how the pricing compares to Brave's search API.
  • asjq178
    Cloudflare protects against bots, Cloudflare sells out its customers to AI scrapers.MITM service, Internet gatekeeper and robber baron.
  • Herz
    How does Cloudflare manage to hit the HN front page almost daily? Don't get me wrong, they build cool stuff, but the frequency is wild.
  • freakynit
    Tried one query on ceramic.ai (the default provider for cloudflare web search api): "qwen-3.8 flash next and rtx 5090 best inference setup" ... 0 results ... same query on google and ddg both yield proper results.Then shortened the query to just "qwen-3.8 flash next" ... results came.. all unrelated. In fact, these were almost all paper links .... no relation to actual search term.And I had thought that I finally had found a cheaper search alternative.
  • kobieps
    so this is like a competitor to https://parallel.ai/ ?
  • measurablefunc
    Very cool.
  • maelito
    Same as https://staan.ai, the European index.
  • daft_pink
    Wow, will they offers some sort of reduce a web page into a markdown file api as welL so we can get full web pages at reduced token sizes?
  • fnordsensei
    I've been quite satisfied with Kagi[1]'s API.1: https://kagi.com/api/docs/openapi
  • babelfish
    What value does Cloudflare provide over using the linked providers directly...?
  • 0fflineuser
    I am pretty sure exa specifically say it trains on your data in it's privacy policy, so how can it be ZDR ?I remember as I was looking at the available web tools for hermes agent not to long ago and looked through the keyless web providers privacy policies, which exa is one of them.
  • tom1337
    I wonder if the three search engines get access to cloudflare protected sites without any captcha or bot interventions
  • 8bite
    The pricing is so different between these:ceramic.ai - $0.25 per 1,000 requestsExa - $7.00 per 1,000 requestsLinkup - $5.00 per 1,000 requestsDoes anyone have insights on the quality differences? Web search API pricing for AI agent usecases has always felt so expensive for what it is, but I have no grounding on the economics of running a web index.EDIT: formatting
  • hmartin
    I've been working on a TypeScript package to provide a unified search API across these providers:https://github.com/hbmartin/agent-web-searchSo this gives a unified search experience without adding another cloud hop and dependency.
  • zergrush
    is this serpapi ?
  • saltysalt
    Wow I guess I am in the minority of folks building a search engine for humans now, this is a wild business model but best of luck to the 3 search index providers sitting behind this proxy, I hope it's worth their while financially speaking. Building an index is hard and expensive (I know).
  • ronfriedhaber
    Interesting to see how this can be compared with Exa, Alas, Cloudflare really is shipping many great orthogonal products recently.
  • anon373839
    > All three support Zero Data Retention for requests made through CloudflareBut does CloudFlare itself commit to zero data retention? If not, this isn’t too meaningful.
  • 1vuio0pswjnm7
    Explore Cloudflare's services without the web page bloat:https://developers.cloudflare.com/llms.txtAs a textmode command line and text-only browser user this textfile is easier for me to use that the usual Silicon Valley style web pagesNot quite as good as sitemap-0.xml but it's nice to have this in addition
  • Oras
    Weird choice by CloudFlare, would been great if they have shared why it was created.I use CloudFlare developer platform and quite happy with tools, but I didn’t use the gateway API and always used OpenRouter which does support web search.I can see it useful for those who didn’t do any integrations or like to keep logs at one place, but did customers actually ask for this?
  • 0xbadcafebee
    SearXNG works pretty well for my personal agents. FYI it's a free search gateway you can host locally, and there are many public instances. It's like the old days when many different people provided the same free service for all.
  • vscarpenter
    I guess I'm not understanding the value here - to compete with Google and the likes, the scale, cost and complexity would be huge. Appreciate new entrants in an existing field but not seeing this one.
  • corentin88
    Funny how search was a graveyard for startups for almost two decades. And since ChatGPT releases (or so) it’s a trending place again.
  • Jeeetendra
    the zero-retention promise from the search providers is useful, but the requests also show up in gateway logs. can you keep the billing data without storing the actual search queries?
  • timpera
    Interesting to see that they didn't include the Brave Search API, which is really great and imo a better experience than Exa.The absence of the Perplexity Search API is to be expected though, knowing how much these two companies despise each other.
  • okokwhatever
    Who thought 5 years ago that searching online would have a cost...
  • OutOfHere
    This has got to be a serious antitrust violation. First Cloudflare bans all the other bots, then it allows its liaised bots.
  • jew-and-proud
    [flagged]
  • trysyncpage
    [flagged]
  • ihsw
    [dead]
  • AlphanimbleAI
    [flagged]
  • irregularbowels
    [dead]
  • agenttavern
    [flagged]
  • locitra
    Search is an interesting building block for agentic systems. The challenge isn't just retrieving results, but deciding what to search for, evaluating the results, and determining when the information is sufficient to move to the next step.As AI systems increasingly use search as a tool, the quality and reliability of that tool become an important part of the overall agent workflow.