Saturday 21 September 2013

Storing Photos In The Cloud And Offline

These days it is easier than ever to take lots of photos. The tricky part is how to sensibly manage all of these photos. I have an existing collection of ~20GB of photos and I take new photos on my iPhone.
 

I want to be able to:
  • Manually organize my photos into albums.
  • Access all my photos from my iOS devices (iPhone/iPad) and on the web - and the cloud storage shouldn't cost too much!
  • Store selected albums offline on my devices for fast access without waiting for each picture to download.
  • Easily sync photos/albums from the cloud down to my PC for the purposes of local backup.
Up until recently I was managing this by using Picasa Web as my cloud store and the excellent Pixite app "Web Albums" on my iPhone/iPad to handle uploads and local album caching. This worked great up until recently when a problem with getting my photos back out of Picasa Web caused me lose my trust in the service. This also made me realise that my current approach didn't include an easy way to sync all of my photos to my PC to protect me from cloud service issues.

Looking At The Alternatives
I have spent some time browsing through existing reviews of the alternatives and so far I haven't managed to find anything which quite fits my requirements.

Offline Caching
The easiest way to filter out much of the competition is to look at the support for offline caching within their iOS apps.
  • Pixite - developers of similar iOS apps for several cloud services which all support offline caching:
    • Picasa Web
    • Dropbox
    • Flickr
  • Everpix - automatically syncs some photos locally. No control over exactly which photos.
I also tried the following apps which didn't meet my requirements:
  • Google Cloud Drive - allows individual files to be cached for offline access. This is useless when I want to make whole albums available offline.
  • Amazon Cloud Photos - no support for organizing photos into albums.
  • No offline caching support:
    • SkyDrive
    • SugarSync
    • Picturelife
    • Loom
So the candidates at this point are (ignoring Picasa Web which I am trying to move away from):
  • Dropbox
  • Flickr
  • Everpix
Uploading Photos From My PC
I currently prefer to organize my photos into folders on my PC. Any service which I choose to use needs to make it easy to upload my existing collection of photos in folders.

Flickr quickly fails this test as the desktop uploader wants to upload a flat stream of photos rather than folders of photos. The closest Flickr equivalent of a folder is a "photoset" but these have to be created manually during upload - not exactly the easy sync I was hoping for! There are independently developed tools available but none of them seemed to offer exactly the kind of sync which I was looking for. I was also put off the Flickr service which seemed complex to setup for storing photos privately - it feels very similar to Facebook in that all of the defaults want you to share your photos!

Dropbox ticks the box for ease of upload as it has an excellent desktop sync client that will simply upload my existing folders of photos.

Everpix provides a desktop uploading client and allows manual re-download of photos. However photos can't be manually organized into albums. I also found that it took a few minutes for each uploaded photo to become accessible. Together these limitations make Everpix unusable for me.

Affordable Storage
Dropbox charges $100/year for 100GB.

I currently have a grandfathered Google storage plan - $5/year for 20GB. However if I want more space the closest normally available plan on Google is $5/month ($60/year) for 100GB.

It is worth mentioning that Amazon Cloud Drive actually has low prices at various capacities - 20GB for $10/year, 50GB for $26/year and 100GB for $52/year (all these prices are approximate as I have converted from GBP). What a pity that the Amazon Cloud Drive Photos iOS app is so poor.

What About iCloud?
As far as I'm concerned iCloud is a total failure for photos. iCloud is not a cloud photo store. iCloud Photo Stream is a cache of recent photos which makes it simple to push photos from the device where you take them to all of your other Apple devices.

Verdict
I already have a total of 36GB of storage with Google at a price I am very happy with. They even have a service (Picasa - becoming Google+ Photos) dedicated to photo management and a 3rd party app (Web Albums by Pixite) which provides offline caching. Unfortunately I just don't trust them to reliably store my photos. I am also frustrated to see Picasa Web redirecting to Google+ Photos which lacks simple features like sorting my albums by their names.

Google Drive would be a great alternative that uses the same storage but is run by a different team in Google that might do a better job with my files. Unfortunately there is currently no way for me to cache entire folders stored in Google Drive offline on my iPhone.

Dropbox seems to be the clear winner amongst the competition as it also provides an easy to use file syncing desktop app and allows me to continue to use a Pixite app on my iPhone to manage my uploads and local caching.

I'll give it a few weeks before I actually move all of my stuff. If Picasa starts working again in that time I might consider sticking with that service a bit longer and just manually download newer albums periodically to backup locally. It is also possible that Google Drive will add folder level caching at some point.

Tuesday 17 September 2013

Why You Shouldn't Trust Picasa To Store Your Photos

tl;dr
A while ago I uploaded this picture to Picasa:


Recently I tried to download this photo and got this back from Picasa:

WTF!

Update: I Found A Workaround!
It turns out that Google Takeaway provides the ability to download individual albums/all photos from Picasa Web. I tried downloading one of my albums which appeared corrupted through Picasa Web/Picasa for Windows and it worked! All of the photos in the album were downloaded correctly! I will probably use this to extract all of my photos and move to another cloud photos service.

Update 2: I managed to extract all my photos using Google Takeaway. 17 photos failed to download and Google Takeaway generated an error report which included links that I could use to manually download the missing photos.

Update 3: This bug appears to be fixed now.

Background
For the last two and a half years I have been uploading all of my photos to Picasa. I also have four years of even older photos which I never got round to uploading to Picasa. I always felt guilty about this as I knew I was taking a risk - that my hard drive could fail and I would lose loads of photos. I actually have multiple local hard drives with copies of the photos but what if my computer was stolen or my house burnt down?

I decided to finally sort this out and get all of my photos into the cloud. I loaded up Picasa for Windows and got it to index all of my local photos. At the end of the process it offered to tag all the faces it had detected in my photos. This sounded like it could be useful but I figured it would be even better to download my more recent photos and tag everything at the same time.

So I ran an Import to start downloading my cloud albums. The first album which downloaded was very large with 316 photos and it was immediately obvious in Picasa that some of the photos weren't loaded properly. I took a look on disk and found that 37/316 photos had been corrupted in the way shown above.

That's ~12% corruption in a single album!

Investigating The Issue
I earn a living as a software engineer so I immediately set to work trying to come up with a meaningful set of symptoms to report to Picasa. This is what I found:
  • The corruption is probably just a truncation:
    • Every corrupted file was very close to (but not exactly) 2MB.
    • The missing data was always at the "bottom" of the image.
  • The original file data is probably not lost:
    • I can actually download uncorrupted individual files using Web Albums on my iPhone. I don't think these are coming from a local cache but I'm not certain.
  • The corruption isn't random:
    • The same files in an album are always truncated when downloaded.
  • It isn't related to the image contents:
    • If I re-upload an uncorrupted image (extracted using my iPhone) then it can be downloaded again without corruption.
  • Picasa might be doing some kind of backend migration:
    • I saw two filename formats within my albums:
      • IMAGE_[0-9]+.jpeg (e.g. IMAGE_141.jpeg)
      • IMAGE_[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+.JPG (e.g. IMAGE_520FFAF1-2FF7-4199-82F4-5A88F5BA8076.JPG)
    • All of the corrupted images have the second filename format.
  • The corruption occurs regardless of whether I download using Picasa for Windows or Picasa Web (tested in IE/Chrome/Firefox).
Reporting The Issue
Time to report the issue! I headed over to Picasa support and took a look at my options which turn out to be pretty limited:
  • Picasa Google Groups - user-to-user support
  • Google+ Feedback - "Although we're not able to personally reply to you, we'll use your feedback to help us continue to improve the Google+ experience."
In other words there is no real way to raise an issue and get a reply.

My issue is with Picasa and not Google+ so I decided to try out the Picasa Google Group but got no reply.

What Now?
Frankly, I am hoping that writing this blog post might get the issue some attention. I am also curious to find out whether anyone else has hit this issue. Maybe I'm the only one?

It seems to me that the Google support model only works when every issue affects enough users to show up in some kind of top level metrics. If Google have really lost a load of my photos I will be really upset but maybe Google doesn't care if there really is only one of me.

The Competition
This issue has made me revisit my decision to use Picasa. The world of cloud photos has moved on in the last few years and maybe I would be better served by another service.

I currently pay Google $5/year on a grandfathered storage plan to get an extra 20GB/year for a total of 36GB. The cheapest up to date plan seems to be 100GB for $5/month.

I took a look at Flickr today and their service offering sounds pretty good - 1TB of storage for free in exchange for seeing some ads on their site sounds great. If I don't want ads I have to pay $50/year which is still cheaper than the cheapest Google plan. They even have a form where you can submit a message and they say they will actually send you an answer!