HN

Need help?
<- Back
Gemma 4 on iPhone

Gemma 4 on iPhone

janandonly

Comments (176)

  • pmarreck
    Impressive model, for sure. I've been running it on my Mac, now I get to have it locally in my iPhone? I need to test this. Wait, it does agent skills and mobile actions, all local to the phone? Whaaaat? (Have to check out later! Anyone have any tips yet?)I don't normally do the whole "abliterated" thing (dealignment) but after discovering https://github.com/p-e-w/heretic , I was too tempted to try it with this model a couple days ago (made a repo to make it easier, actually) https://github.com/pmarreck/gemma4-heretical and... Wow. It worked. And... Not having a built-in nanny is fun!It's also possible to make an MLX version of it, which runs a little faster on Macs, but won't work through Ollama unfortunately. (LM Studio maybe.)Runs great on my M4 Macbook Pro w/128GB and likely also runs fine under 64GB... smaller memories might require lower quantizations.I specifically like dealigned local models because if I have to get my thoughts policed when playing in someone else's playground, like hell am I going to be judged while messing around in my own local open-source one too. And there's a whole set of ethically-justifiable but rule-flagging conversations (loosely categorizable as things like "sensitive", "ethically-borderline-but-productive" or "violating sacred cows") that are now possible with this, and at a level never before possible until now.Note: I tried to hook this one up to OpenClaw and ran into issuesTo answer the obvious question- Yes, this sort of thing enables bad actors more (as do many other tools). Fortunately, there are far more good actors out there, and bad actors don't listen to rules that good actors subject themselves to, anyway.
  • karimf
    This app is cool and it showcases some use cases, but it still undersells what the E2B model can do.I just made a real-time AI (audio/video in, voice out) on an M3 Pro with Gemma E2B. I posted it on /r/LocalLLaMA a few hours ago and it's gaining some traction [0]. Here's the repo [1]I'm running it on a Macbook instead of an iPhone, but based on the benchmark here [2], you should be able to run the same thing on an iPhone 17 Pro.[0] https://www.reddit.com/r/LocalLLaMA/comments/1sda3r6/realtim...[1] https://github.com/fikrikarim/parlor[2] https://huggingface.co/litert-community/gemma-4-E2B-it-liter...
  • PullJosh
    This is awesome!1) I am able to run the model on my iPhone and get good results. Not as good as Gemini in the cloud, but good.2) I love the “mobile actions” tool calls that allow the LLM to turn on the flashlight, open maps, etc. It would be fun if they added Siri Shortcuts support. I want the personal automation that Apple promised but never delivered.3) I am so excited for local models to be normalized. I build little apps for teachers and there are stringent privacy laws involved that mean I strongly prefer writing code that runs fully client-side when possible. When I develop apps and websites, I want easy API access to on-device models for free. I know it sort of exists on iOS and Chrome right now, but as far as I’m aware it’s not particularly good yet.
  • janandonly
    OP Here. It is my firm belief that the only realistic use of AI in the future is either locally on-device for almost free, or in the cloud but way more expensive then it is today.The latter option will only bemusedly for tasks that humans are more expensive or much slower in.This Gemma 4 model gives me hope for a future Siri or other with iPhone and macOS integration, “Her” (as in the movie) style.
  • gdzie-jest-sol
    I need normal server too in local network I can run chat in other device and 'counting' on iphone.Second idea is input audio in other language, like Czech, Polish, French
  • jeroenhd
  • al_borland
    I find it odd they are using the term “edge” to brand this, if it’s target is the general public.I’ve been to a few tech conferences and saw the term used there for the first time. It took me a little bit to see the pattern and understand what it meant. I have never heard the term used outside of those circles. It seems like “local” would be the term average users would be familiar with. Normal people don’t call their stuff “edge devices”.
  • totetsu
    I have been looking at ARGmax https://www.argmaxinc.com/#SDK for running on apple devices, but not sure yet at whats involved in porting a model to work with their sdk
  • rock_artist
    I really believe in the future of local models.From app developer and user, My main concern for now is bloating devices. Until we’ll have something like Apples foundation model where multiple apps could share the same model it means we have something horrible as Electron in the sense, every app is a fully blown model (browser in the electron story) instead of reusing the model.With desktops we have DLL hell for years. But with sandboxed apps on mobile devices it becomes a bigger issue that I guess will/should be addressed by the OS.For my app I’ve been trying to add some logic based on large model but for bloating a simple Swift app with 2-3GB of model or even few hundred MBs feels wrong doing and conflicting with code reusability concepts.
  • dhbradshaw
    My son just started using 2B on his Android. I mentioned that it was an impressively compact model and next thing I knew he had figured out how to use it on his inexpensive 2024 Motorolla and was using it to practice reading and writing in foreign languages.
  • _nagu_
    If this works smoothly on iPhone, it could change how we think about mobile apps. Less backend dependency, more on-device intelligence.
  • allpratik
    Nice! Tried on iPhone 16 pro with 30 TPS from Gemma-4-E2B-it model.Although the phone got considerably hot while inferencing. It’s quite an impressive performance and cannot wait to try it myself in one of my personal apps.
  • deckar01
    It doesn’t render Markdown or LaTeX. The scrolling is unusable during generation. E4B failed to correctly account for convection and conduction when reasoning about the effects of thermal radiation (31b was very good). After 3 questions in a session (with thinking) E4B went off the rails and started emitting nonsense fragment before the stated token limit was hit (unless it isn’t actually checking).
  • hadrien01
    Is it me or does the App Store website look... fake? The text in the header ("Productiviteit", "Alleen voor iPhone") looks pixelated, like it was edited on Paint, the header background is flickering, the app icon and screenshots are very low quality, the title of the website is incomplete ("App Store voor iPho...")
  • TGower
    These new models are very impressive. There should be a massive speedup coming as well, AI Edge Gallery is running on GPU, but NPUs in recent high end processors should be much faster. A16 chip for example (Macbook Neo and iphone 16 series) has 35 TOPS of Neural Engine vs 7 TFLOPS gpu. Similar story for Qualcomm.
  • burnto
    My iPhone 13 can’t run most of these models. A decent local LLM is one of the few reasons I can imagine actually upgrading earlier than typically necessary.
  • rcarmo
    This is fun. I just wish I could add more skills, the UX is too dumbed down but knowing there is a run_js tool there is a lot that can be done here.
  • two_handfuls
    The description says it's private, but the legalese it makes you agree to makes no promise. Rather, the opposite:> We collect information about your activity in our servicesSource: https://policies.google.com/privacy#infocollect
  • rudedogg
    This is fun, FYI you don’t have to sign in/up with a Google account. I hesitated downloading it for that reason.
  • danielrmay
    I spent some time getting Gemma4-e4b working via llamacpp on iPhone and I'm really impressed so far! I posted a short video of an example application on LinkedIn here https://www.linkedin.com/feed/update/urn:li:activity:7446746... (or x: https://x.com/danielrmay/status/2040971117419192553)
  • carbocation
    It would be very helpful if the chat logs could (optionally) be retained.
  • MysticOracle
    Crashes for me on a couple of different iDevices (2 generations behind) after only a few 2-3 chats. Probably not enough RAM.Saw this one on X the other day updated with Gemma 4 and they have the built-in Apple Foundation model, Qwen3.5, and other models:Locally AI - https://locallyai.app/
  • davecahill
    I really like Enclave for on-device models - looks like they're about to add Gemma 4 too: https://enclaveai.app/blog/2026/04/02/gemma-4-release-on-dev...
  • satvikpendem
    This is also on Android and has an option to use AICore with the NPU which can run much faster than even the GPU models.
  • dwa3592
    I think with this google starts a new race- best local model that runs on phones.
  • neurostimulant
    I'm able to sweet talk the gemma-4-e2b-it model in an iphone 15 to solve a hcaptcha screenshot. This small model is surprisingly very capable!
  • XCSme
    Gemma 4 is great: https://aibenchy.com/compare/google-gemma-4-31b-it-medium/go...I assume it is the 26B A4B one, if it runs locally?
  • jdthedisciple
    it's Google, so is it really private?remember, megacorps are dying for infinite amounts of analytics data
  • nickvec
    Extremely impressed by how fast responses are on iPhone 17 Pro Max. Can’t wait for this to be used for Siri’s brain one of these days (hopefully!)
  • rotexo
    E4B is pretty good for extracting tables of items from receipt scans and inferring categories, wish this could be called from within a shortcut to just select a photo and add the extracted table to the clipboard
  • Sharmaji000
    Still didnt release training recipe, data, methodology etc unlike deepseek. Mostly released to get developer ecosystem across their android built in ai. Still good and interesting, but not exactly philanthropic to the open source progress.
  • thot_experiment
    Gemma 4 E4B is an incredible model for doing all the home assistant stuff I normally just used Qwen3.5 35BA4B + Whisper while leaving me with wayy more empty vram for other bullshit. It works as a drop in replacement for all of my "turn the lights off" or "when's the next train" type queries and does a good job of tool use. This is the really the first time vramlets get a model that's reliably day to day useful locally.I'm curious/worried about the audio capability, I'm still using Whisper as the audio support hasn't landed in llama.cpp, and I'm not excited enough to temporarily rewire my stuff to use vLLM or whatever their reference impl is. The vision capabilities of Gemma are notably (thus far, could be impl specific issues?) much much worse than Qwen (even the big moe and dense gemma are much worse), hopefully the audio is at least on par with medium whisper.
  • modeless
    It's so ridiculous that Google made a custom SoC for their phones, touting its AI performance, even calling it Tensor, and Apple is still faster at running Google's own model.Google really ought to shut down their phone chip team. Literally every chip from them has been a disappointment. As much as I hate to say it, sticking with Qualcomm would have been the right choice.
  • mc7alazoun
    Would it work locally on a Mac Pro M4 24gb? If so I'd really appreciate a step-by-step guide.
  • anon
    undefined
  • rickdg
    How do these compare to Apple's Foundation Models, btw?
  • Waterluvian
    I see a phenomenal opportunity for old phone re-use by arraying them in some dock and making them be my "home AI."
  • tithos
    Most of the models are not available. I’m guessing they will become available soon enough… At least I hope.
  • garff
    How new of an iPhone model is needed?
  • beeflet
  • dzhiurgis
    I recently got to a first practical use of it. I was on a plane, filling landing card (what a silly thing these are). I looked up my hotel address using qwen model on my iPhone 16 Pro. It was accurate. I was quite impressed.After some back and forth the chat app started to crash tho, so YMMV.
  • meidad_g
    [dead]
  • micmcfly
    [dead]
  • darshil2023
    [dead]
  • LeonTing1010
    [flagged]
  • lol8675309
    It’s gotta be free!?!? Right!?!? Oh oh wait
  • __natty__
    That's a great project! I just wondered whether Google would have a problem with you using their trademark
  • yalogin
    Are these models open source? If so this is Google’s attempt to collect user data from their models.