Apple Photos app corrupts images

<- Back

Apple Photos app corrupts images

pattyj

Comments (382)

deviation
It seems to be an import pipeline bug.Photos does a lot of extra work on import (merging RAW+JPEG pairs, generating previews, database indexing, optional deletion), so my guess is a concurrency bug where a buffer gets reused or a file handle is closed before the copy finishes.Rare, nondeterministic corruption fits the profile.
CharlesW
"I don’t know if this is a problem that is specific to OM System cameras, and I’m not particularly interested in investing in a new camera system just to find out."A better way to have further narrowed down the problem down to Actually iPhoto would've been to do the same tests with a USB-C card reader plugged directly into the Mac, which would've eliminated cables, hubs, and camera hardware/software/firmware as possibilities.It's worth noting that searches show that OM-1 USB support is imperfect, the camera manual addresses that "USB transfers aren’t guaranteed in some setups", and user consensus seems to be to use a card reader for reliable file transfers.
drnick1
You couldn't pay me to go back to using anything made by Apple or Google. I use a GrapheneOS Pixel and my self-hosted "cloud" with Nextcloud, HomeAssitant, and my own email server and the control and performance are unrivaled by any of the Big Tech crapware. I could start self-hosting an AI chatbot I suppose, but these do not seem to have reached the point of enshiffication yet as relatively new services.
doodaddy
As an Olympus shooter this is good to know.But good gravy that troubleshooting path got expensive real fast. Replacing the laptop and the camera? Why not start by trying something other than Photos? It doesn’t even need to be a paid product; the Olympus software is free not to mention a good baseline since it - of all the applications - should be able to import photos without corrupting them.Edit to add: delete on import seems pretty risky. My workflow is to import and only delete from the camera after 1) the imported photos are backed up 2) I’ve done a first pass culling.
ChrisRR
I feel like this is a URL that I don't want in my history
asolove
I also have an OM System camera (OM-5) and never get corruption this bad but occasionally got one row of green pixels at the bottom of a photo during import to Photos. I thought I was crazy, but this motivates me to change up my routine and check if it was Photos all along.
elliotec
Always count on Tenderlove for a detailed technical deep dive! I've missed your blog.Tangential story - 12-13 years ago I was a burgeoning and super eager software dev that moved to Seattle to be closer to "the scene." tenderlove's content was a major reason for me going there and I poured through his posts learning way too much about Nokogiri, Active Record, and much much more.I went to every Ruby meetup I could get to out there and I remember one in particular, a Seattle RB meetup, in the Substantial office. It was a pretty small group, at most 15-20 people.I was with a coder buddy but knew nobody else. We were all just drinking pints of Manny's beer and eating pizza from Big Mario's or something. Ryan Davis (the creator of minitest among other things) was doing a presentation on Unicode.Aaron Patterson (tenderlove) was cracking jokes at every opportunity. At one point I asked a relatively naive question and Aaron _tenderly_ answered in joke-form response. I felt such a _part_ of the scene then. Aja Hammerly was super engaged in the presentation, I think even Ryan Bates and/or Geoffrey Grosenbach were there.It was quite surreal to be in this dream-like state around giants and heroes just doing what they were doing and being so inclusive. It seemed so normal but became a core memory.Thanks for everything Aaron, you've truly been an inspiration!
watersb
iTunes plays audio files. It also has metadata for the audio: Artist, Title, album art, how many times a song has played...It writes the metadata into the audio file. Badly.In particular, it's that "Play Count": iTunes rewrites the audio file every time you play the song.Usually it just corrupts the metadata enough to forget the album art. But it's perfectly willing to destroy the audio data.
MarkMarine
I’ve been seeing this happen on older photos that had imported properly, and I just use my iPhone and view photos on my Mac and iPhone. Looking back, I’ve lost whole chunks of my photo library. It’s a bigger problem than I realized. I don’t have these backed up elsewhere.
sitharus
I hadn’t dug that far in to it, thanks for sharing! I assumed my rather old SD card or the adapter I keep stuffed at the bottom of my bag was the issue as I’ve only seen it on a couple of photos.I’ve used Olympus cameras for over a decade. Well, the same camera to be honest, a PEN E-PM2. This has only appeared in the past couple of years.I haven’t seen it on photos from my Canon EOS 80D yet, but I guess it’s time to change my workflow. And maybe OS.
spike021
I realize it's easy to think or assume this is a bug affecting everyone but for what it's worth, I've had zero issues importing photos from my Sony a6000 and a7iii for the past eight or so years. Tens of thousands of photos at this point.For the longest time my process while traveling was importing onto my iPad or occasionally my iPhone since I didn't have a personal laptop, just a Mac Mini at home.
itake
Apple corrupted images on my iPhone where I can’t import them to my PC via photos, but I can backup the whole phone.They finally recognized there is an issue, but there is no fix, as of a few weeks ago :(
myshkin5
I’m a fan of the whole Apple ecosystem but I have to say that there’s a pattern here. Apple does a decent job of keeping my data safe from others but a terrible job of keeping it intact. From music libraries with song titles that got switched to long integers to this (and I’m sure more that I’m not remembering atm) they need to do a better job here.Sure security is important but integrity is too.
ByteDrifter
I’m not a pro, but I use iCloud Photos to back up everything. This bug honestly makes me nervous. If something gets corrupted and I don’t even notice I might lose years of memories without knowing.
spuz
It would be very helpful to document the version number of the Photos app that demonstrates this behaviour so anyone else who is affected can use this article to keep track of potential fixes.
dzink
Not sure if related but importing images via image capture on mac to the disk of the mac gives you correct time when the photo was taken in the file (kind of important if it’s family photos). But if you import it to a usb drive you get current time as creation time for each file so you’ve lost any timestamp you had on the photos.
BrtByte
Kind of horrifying. You expect Photos to be dumb and slow, sure, but corrupting files
hosteur
I am not letting Apple Photos touch my photos. Neither Google Photos, etc.All my photos are managed using Digikam and developed using Darktable. They are also visualized via immich, but immich only has access via a read-only mountpoint.Everything is hosted locally of course.
driggs
I've not experienced corruption like the author, since my workflow involves copying the raw files from sdcard to harddrive, and then importing into Photos. After processing the raws in Photos, I export a .jpg back out to the filesystem.That's because my worry is corruption of the entire Library, which Photos stores as one gigantic opaque file/directory abomination. My .photoslibrary file is currently 70gb in size, and I'm terrified of what would happen if it becomes corrupted. The Photos app crashes not infrequently.
giancarlostoro
> Turns out “delete after import” was a huge mistake.https://www.cgsecurity.org/wiki/photoRec
roc856a
Your workflow was horrendous, and now it's merely bad. Don't touch any images on the card until you're sure that the images imported correctly AND your local and cloud backups have backed up the images. I assume you have local and cloud backups. If not, you should set them up right away. Really, cards hold a huge number of images and there should be no rush to empty them out.
xlii
I feel it's OM camera issue.I've been importing raws for years from Sony and earlier from Pentax and didn't experience it.In fact searching for "OM image corruption" shows bunch of results not related to Apple Photo.My guess is that OM has buggy SD driver which starts deletion before actual read finished.
zahirbmirza
I have had raw files on an CF card that were corrupted by an import attempt. I never used iOS import again unless the card was backed up to a computer first.Recently, importing via image capture has resulted in recurrent crashes. Files appear on Image Capture that do not appear on the iPhone, nor can they be downloaded via image capture, or deleted. I wish I knew wtfudge was going on.
zenmac
Noticed that quite bit on old backups of large libs. Being just using Image Capture and backing up to filesystem!!! HD is probably a better medium time backup than some proprietary photo app.
dkga
Curiously, to me at least the image of the corrupted file was itself corrupted and not shown by my browser. Or maybe Apple didn't want me to see the corrupted file... :)
amatecha
Wow, I wonder if the same issue exists in the "Image Capture" app which I've used for years on my old MacBook to transfer photos off my iPhone. I would transfer everything off and occasionally find that some photos are inexplicably corrupted. The app itself is terrible, it regularly shows it imported an image but didn't, or it arbitrary imports a couple images completely randomly NOT from the ones I selected, so I end up with weird duplicates and stuff. :\
poolnoodle
Apple just isn't as good at software anymore. The same thing can be said for almost any big software company, though. Generally QA seems to really not be a priority anymore.
tmountain
I shoot RAW but I wouldn't want to eat up all my iCloud space with my RAW files. They're 80MB each off of my Fujifilm camera. I store them on a local DAS instead. Curious what the real use case is for storing RAW on iPhoto.
yard2010
> Turns out “delete after import” was a huge mistake.I learned the hard way to never delete photos from the SD. Just buy a new one it's so cheap anyway.Great article by the way, sounds like my kind of rabbit hole :)
trevorkoob
I've stopped using photos entirely due to import bugs.So much wasted time, now I just use image capture to import and organize directly.Still get errors from time to time.Apple needs to hire more quality control, their software integration is going from a positive to a negative.Seamless integration was a large part of Apple's initial hook, and continues to be a part of their drive to push services, it should be a priority.
mcflubbins
What are some good Open Source / Self Hosted alternatives to Apple Photos (Desktop)? I pretty much keep my Mac Mini around solely to import photos from our phones, free up space on the phone, and backup the Photos DB. We like to go back and look at old photos from time to time too, and the feature that shows them on a map is a big one for us.Last time I looked (pre-COVID) there wasn't a lot of promising options, and some didn't support HEIF images
hn111
I remember years ago I lost some random videos on iCloud (synced via the Photos app). The filenames were there but they couldn't be opened anymore and seemed corrupt. Since then I never trusted iCloud anymore and switched to Syncthing (which back then also messed up some of my data if I recall correctly) and then Nextcloud.
jcbe
I have tried fairly persistently to make Apple Photos my primary photo management tool and I finally gave up recently. The app crashes repeatedly when I have it open—only occasionally until this year but the frequency has increased to the point that it no longer feels usable. A real shame. I’d rather stay in the ecosystem if I had the choice.
mcflubbins
I had a weird issue with at least one photo in Apple Photos recently (possibly more that I haven't found) where the photos app showed the image, but I couldn't export it - like it was only a preview. I've upgraded my photos database over many release so I don't know if that's a part of it, the photo in question was from 2018 or so
intrasight
So restore your original images.
larusso
Back in 2011 I did the grave mistake of updating my iPad to the beta version of iOS. It was iOS 4 I believe. I took it with me on my honeymoon travel in US. My use case was to offload images onto the iPhotos app with an SD adapter. I bought the Apple Dock one.On day 7 or so the import failed and all files on the pad got corrupted. But also the SD card got corrupted.I stopped using the device and the card because I knew not all is lost. I had to buy a new card in SF as replacement. Back home I used a recovery software to check if data is still on the card (I used the same software before on a card that got deleted by another person and I was able to get all images back). I was able to get most of the images recovered and also recovered a few from the iPad. All in all I lost maybe 10 out of a few 100. Now I travel with multiple cards and backup already each night while in the hotel. And I don’t delete the images on the SD Card. I format only when I’m sure I have everything copied and secured.
mulmen
I recently bought a house and the seller included some slides from when he bought the house in 1993. They are still clear and visible, even without a projector. I highly doubt I will be able to share my iCloud photos in 2057.
Retinal7467
This is really good to know. I’ve never personally run into this problem despite having the same hardware because I only ever put the final jpeg into apple photos and keep the camera output completely outside of the Apple ecosystem on a hard drive.
jamesgasek
The recent degradation of Apple software quality is crazy
lo_fye
Have you tried importing them using the Image Capture app on iOS, instead of the Photos app? It just gets them off the camera/SDCard and onto your Mac in a folder, which you can then drag onto Photos.app -- worth a shot.
maz1b
I stopped using the Photos App on Mac because of this, has happened on several occasions.
LeoPanthera
Changing the end of this headline to "corrupts some images imported from external cameras" would be longer but a lot less clickbaity.
bichiliad
Somewhat tangental, but I keep my music in the Music app. Wireless music sync is great and usually does what I need. Once in a blue moon, however, it'll absolutely scramble every album cover of every song I have.
tehlike
Past few days, i created a copy of all my photos on google in my nas. This gave me a peace of mind in case something catastrophic happens (image corruption, account getting banned etc)...
anon
undefined
anArbitraryOne
It comes back to my fundamental philosophy that apple is the problem and we should abandon it until it shapes up
CtrlAlt
I’ve never had this bad of corruption. But not surprised.Personally, I have seen a row of green pixels on the top or bottom + vertically flipped photos on import.Good sleuthing!
ImPrajyoth
It feels less of security related and and more of ecosystem lock-in related to be honest
locao
I'm feeling dumb, but that seems a really different RailsConf than I was expecting.
wormius
And Absolute Apple Photos app corrupts images absolutely.(sorry not sorry)
stmw
"Mr. Jobs reportedly asked the assembled engineers and other MobileMe team members, “Can anyone tell me what MobileMe is supposed to do?” When one of those employees then volunteered a satisfactory answer, Mr. Jobs followed up with, “So why the fuck doesn’t it do that?”https://www.businessinsider.com/steve-jobs-mobileme-failure-...
theturtle
iPhoto still works just fine, and I refused to move. And I never have it or anything else be the only place an image exists. Storage is cheap.
lapcat
See also the Image Capture bug from several years ago where it appends a ton of empty data to imported photos:https://cdfinder.de/blog/files/image_capture_bug.html(I'm not sure whether this bug has been fixed or not yet, though I think it has been fixed.)
imagetic
Photos has always corrupted stuff.
reboot81
Im interested to see if you encounter the same with a different phone. Most probably a software issue, but are we sure it isnt something defective with your phone?
kokey
Fortunately it mentions early on in the article that this is related to an Olympus camera so I'm guessing this has something to do with the OM system's flavor of Olympus's proprietary ORF format.
nothrowaways
Apple photos also distorts photos. It makes all celebrity selfies look they were driving in the UK.
aakkaakk
This bug has been there for years, probably a decade.I got similar symptoms as mentioned, I suspected AFS+, but what do I know. It has happened on at least 3 iphones (pros), now when I think about it, I don’t remember any iphone I haven’t have troubles with. Having 5000+ images (non-raw) where 5-10% are corrupt is infuriating, but I just stupidly buy another iphone every year (the most expensive one).Re-importing images 10-100 times could sometimes extract a few additional images, but the phone just disconnects after a while when running such scripts.
renewiltord
Use photorec to recover from the old SD card even if you deletedThen for the current files you’ll want to see what happened. Often with this class of problem either the bytes are zeroed or shifted. Since the size is the same, perhaps they’re zeroed or perhaps bytes are LE to BE or dumb shit like that (don’t know why it would be but weird world right).Just diff and see if you see anything (I wrote off memory but you get the idea) diff -aui <(xxd -r file1) <(xxd -r file2) If files are getting zeroed sucks but otherwise maybe you can swizzle it back out. If full bytes look weird, look at binary representation and see if you have pattern.From that bare start you can see what’s up.
pandemic_region
> Bought a new laptop Bought a new camera: the OM System OM-1 MKiiI think he was just looking for an excuse to buy new kit
cmurf
The default behavior for handling cards containing images is a read-only mount, and copy all the images without any modifications.When flash fails it returns garbage or zeros instead of (what was) your data. It can be tranient or persistent. And without any error codes from the storage device or the file system.If storage returns garbage for filesystem metadata, all bets are off how the OS filesystem driver will behave.Reformat should be done in camera. And that card used only in that camera. And only that camera gets to write to that card. And don't delete individual images.
maipen
In the age of AI, apple should just open source these basic apps. They really don’t have anything to lose if they this.
otterdude
Apple is the new dogshit. Having serious issues with nearly every one of their offerings. Apple TV app needs to be logged in every time. iTunes plays video/audio out of sync. hardware failures on macbook pro / apple watch.I've stopped buying apple stuff
dbg31415
The headline should really note that this only affects people importing from a digital camera.For the 99.999% of us who just use our phone, nothing to worry about.Scared me for no reason! =P
bluSCALE4
I have Apple Photos but I never thought to use it to automatically import my photos and clean it up. My process is very similar to where you've ended up. Thanks for validating it--I'll never change it.
VeejayRampay
for context, the author is Aaron Patterson of Ruby and Ruby and Rails fame, a proficient C programmer and overall hacker, he knows his stuff
anon
undefined
tamimio
For transferring files (photos or others) from iOS, I have been using Landrop for a while and never had any issues so far, it’s also way faster than using a cable.
internetdrew
...just discovered DarkTable because of this! #win
2OEH8eoCRo0
Imagine having the resources of Apple and not testing your software. We deserve better.
huflungdung
[dead]
smeeger
[flagged]
amelius
[flagged]
pvdebbe
[flagged]
outsideoftime
[dead]
kevwil
WTF is that URL? I'm NOT clicking on that. LOL
anon
undefined
billyjobob
He says the checksums are different but he doesn’t provide a diff to show how different. It could just be a single flipped bit or something. And that could happen in his own RAM/disk/CPU/router so seems premature to immediately blame Apple.
anon
undefined
hk1337
> Turns out “delete after import” was a huge mistake.That's a mistake no mater what application you're importing to, else we'll be graced with another blog post, "Darktable app Corrupts Photos".What's the purpose of RAW+jpg though? Seems rather redundant?