November 28, 2012

There’s always a debate on whether it’s better to keep an old car going vs. getting a new one. With an older car, you may have problems with the engine, transmission, but you reassure yourself that it’s cheaper to just fix what you have than to trade up. Fast forward a couple months or years. You’re driving down the highway and the car breaks down, leaving you stranded. And as you stand there, waiting for help, you realize the time has come.

We have our own car here within the Photosmith development effort. For two years now, we’ve been using the model as recommended by Apple for centralized iPad picture storage – the built in Photos.app. The underlying library that gives access to the Photos.app is referred to as ALAssets. And while doing this certainly has its advantages for the workflow (namely, one place for your images), it has caused more problems that can even be described in one blog post. The technical shortcomings, oversights, and pitfalls within this library have translated into inordinate amounts of time and effort in development – we’ve had to work around problems that just shouldn’t exist. We’ve submitted multiple bugs and feature requests to Apple, but the problems persist. And, to top all this off, the ALAssets library alone has the biggest effect on the stability of the app. While we can work around some of the problems, we’re finding ourselves left on the side of the road too many times. The time has come to trade in the car.

And that’s exactly what we’re working on – replacing the link-to-Photos.app with an import process. Changing foundational architecture is always painful, but this is a move that is long overdue. We’ve had a couple reported bugs as well, and will be addressing those with the new development effort.

 

New Import Process

That’s right – we’re revamping the import process. In fact, we’re completely re-working the entire structure of how Photosmith keeps track of images. Going forward, Photosmith will have an actual “Import” dialog, where you can select which pictures you want to pull into the Photosmith sandbox. Once they’re imported, you can do everything you can currently, without the random crashes we’re getting from ALAssets rearing its ugly head. No more waiting for the app to sync on startup every time.

Moving to this model also allows us to re-work the Lightroom sync process. Now the previews will be loaded within the sandbox, and not to the Camera Roll. And if you have an iPad 2/3/4/mini, you’ll be able to import from Lightroom or Eye-Fi and still use the app at the same time (the current popup during sync is a limitation as a result of ALAssets quirks); an iPad 1 doesn’t have the resources to handle the sync and the tagging /viewing of large images at the same time. Also- we’ve been able to re-work sync so that it’s faster, too.

And since we won’t be relying on ALAssets for the images, images will load quicker. Histograms will faster. No more “metadata not yet loaded”.

Preliminary tests are promising. We’ve been able to do 5,000 and 10,000 image syncs from Lightroom, averaging 4 seconds per image upload (about 2x as fast as the current version). The Import dialog is being worked on, and we’ll be able to start processing imports from Photos.app in the next couple days.

And in the event that Apple decides to give developers the ability to download images directly from the camera through the Camera Connection Kit, we’re ready to support that. (Sound nice? We completely agree. Let Apple know.)

 

Sounds Perfect!

Well, this move isn’t without it’s issues. For example, when you delete a picture, what does that mean? Does it delete it in Lightroom or only the preview? What if you’ve imported an original from Photos app? How can you be sure that you’re not about to lose your original data? Should there be a confirmation each and every time? Should there be a trash that you have to empty to confirm?

Every one of these questions may seem obvious at first, but each has it’s own implications logically and with the UI and the workflow that all need to be considered. Yes, you’ll be able to delete, but that delete sword is certainly double-edged.

 

So, When Will It Be Ready?

In the end, we’d much, much rather be working on new features in the app rather than chasing down demons from ALAssets. And we knew that this new development was going take a fair amount of time. We don’t know exactly how much time – probably “a couple months”. We’re determined to get it right and to heavily test, and know this will just take time.  We’ve become very reluctant to release time frames because we just don’t know; what we can do is keep you informed about current progress.

Yes, this will be a free upgrade for all users. We’ll also need to write migration code to move your existing data into the new format.

Right now, we’ve re-designed the database and have started work on the Import dialog and updating the Lightroom sync to support the new system. We’ve started testing core functionality and memory usage for those functions.  We’ve pulled in some additional developer help as well to help us refactor some of our existing code and truly cut the cord with ALAssets (namely – doing all our own raw data parsing and not relying on iOS for anything). Exciting times ahead!

 

 

Posted in: Current Progress | 27 Comments

27 Responses to “Time for an upgrade”

  1. Maxime says:

    Oh man, it’s like I am reading my name all over this update. Thanks guys and take your time to make it work.

  2. Stephen Christy says:

    So far as I’m concerned you build up considerable amounts of trust and respect with the open and frank updates about what is happening. I very much respect both that and what you are doing: thanks so much.

    Stephen

  3. PegaS says:

    I was looking for an iPad review solution on the web and not really sure as this is the right place to ask, but maybe it is.
    In the current (or future revamp) of the version – is there a way to transfer only JPEG previews of the image from Lightroom and not the full image? Maybe even a downscaled version of it?
    My girlfriend, being a wedding photographer shoots quite a bit of images per session (5-7 thousand is normal) would really love to move the reviewing step to the iPad, but copying this many RAW images wouldn’t only take up most of the iPad’s storage, but will be extremely heavy on the processor/RAM while reviewing them. Transferring downscaled JPEG previews would solve this problem.

    • Chris Horne says:

      PegaS:

      Yes, that exact option exists currently in the plugin for the reasons you mentioned. It doesn’t make sense to transfer the full raw image, so the plugin has the option of transferring the fullscreen or fullresolution jpg of any image in Lightroom over to Photosmith. Once the image has been transferred, only the metadata is exchanged for future syncs.

      The current version will accomplish what you’re asking about, although ALAssets will crash before 5-7 thousand images completely upload. The images will be stored in the iPad’s Camera Roll. The updates mentioned above take care of both of these issues.

      Feel free to contact us via the Support page if you have any other questions – we’re happy to answer them.

      • PegaS says:

        Thanks for the info, Chris!
        How far down the road is the update (or do you even know at this point)? I’ll definitely give it a shot once the new version comes out.

        • Chris Horne says:

          It’s too early to tell right now. If we quote a date, it’d be purely speculation, so the best I can tell you is “a couple months” and we’re working as quickly as we can. We know it’s a big deal, but we also want to deliver a good, stable product.

          We’ll be posting updates to the blog – just keep an eye out here.

  4. Jonathan says:

    This sounds great, but will importing from the camera roll into photosmith create a duplicate file (i.e. take up twice the storage space on the iPad)? I guess I could delete files from the camera roll once they’ve been imported, but then they wouldn’t be available to other apps (e.g. snapseed or PS touch).

    I’m a landscape photographer and I spend long stretches out in the field without access to a computer, so space is always a bit of a premium.

    • Chris Horne says:

      You’re right – importing twice eats up valuable space. This was one of our big concerns with going this route. And so we’ll have multiple import options, including the ability to “copy” into Photosmith or “link” back to the original Photos library. For the linked pictures, we’re still moving all the preview data into the sandbox, so that once it’s imported, we don’t have to touch ALAssets again until it’s time to Lightroom sync.

      I personally tend to shoot in the same way you do, in that I take my camera away in the field and so space is always a premium. We have a couple solutions forthcoming. Some will be featured here on the blog in the next few weeks (ie – copy only the previews into Photosmith, but keep the originals on your memory cards and the sync process will put it all back together when needed), and others we can’t talk about just yet. :)

  5. Jim Fawcette says:

    Interesting feature set. Any chance you will support Aperture in addition to Lightroom? Good luck and thx.

    • Chris Horne says:

      There’s a lot of demand for it; it’s been on the list for a while.

      However, it doesn’t make sense for us to divert work on stability to new features right now. First things first…

  6. Dave says:

    I just got my first Eye-Fi card, and I am seeing lots of hangs and/or crashes during transfers from the card to Photosmith. Do I have to wait for the new import process for these to be fixed, or are there any Eye-Fi fixes coming sooner? Or are there any workarounds to make the Eye-Fi to Photosmith transfers more reliable? Thanks for the help!

    • Chris Horne says:

      The new architecture *is* the fix, unfortunately – the crashes are a result of writing out to the Photos app – it’s simply not stable enough for that purpose, despite our best attempts to make it so.

      My only suggestions to make it more reliable is to shoot raw+jpg and send the smallest jpg you can over EyeFi. Photos app seems to be a bit more stable with that.

      • Dave says:

        Hi Chris, I have a Canon 60D with just one card slot. If I shoot raw+jpg, is there a way to just send just the jpg’s to Photosmith and leave the raw’s on the Eye-Fi card? Thanks!

        • Chris Horne says:

          There’s no option that I know of on the 60D. We’re exploring allowing on/off options for jpg and raw, to allow control from the Photosmith side. The split jpg/raw workflow seems to be the most efficient, and I’ll admit we need to do a little bit more to make that smoother; I’ll be addressing that in the code for the next release.

  7. Brian Wong says:

    Guys,

    Super excited about the new direction the product is headed. I think you’re making the right call, even though it will likely be a long, painful process. I’m still rooting for you and PhotoSmith!

  8. Ken Burke says:

    Just wanted to raise my hand and join the numbers of folks who would love to see Aperture support. Know your plate is full at the moment, but wanted to throw my “want” on the pile. Thanks!

  9. George says:

    I am also eagerly awaiting the update as I purchased Photosmith for the sole purpose of importing via my Eye-Fi card and employing the Proxy JPG workflow with LR4. Unfortunately any Eye-Fi transfers crash the app making it unusable for me. I hope you guys can get this worked out soon.

    Thanks!

  10. Ed Letven says:

    Former ad man, now painter. Present my work to galleries on Ipad3. Still struggling with the fact that
    pictures I take of my work or just photography in general is put through old Photoshop 7 to control gamma and correct brightness to equal the painting’s actual saturation. I cannot transfer my “managed” images from my Power Mac to the Ipad without either sending them to myself by email or using Drop Box which is incredibly oblique and confusing program. I tried to copy my jpegs directly to a smart card and using the Ipad’s drive connector and USB cable transfer images to Ipad. Won’t work. It only takes jpgs generated by the camera. Will this new Photosmith Version Whatever be able to help me? I’ll hang up and listen.

    • Chris Horne says:

      Ed-

      If you’re looking for a simple solution, you could try using Apple’s Camera Connection Kit – this would allow you to transfer images from your SD card or USB camera directly into the the Photos app, where you can browse from there.

      Photosmith is designed to work with Lightroom, and you would be able to send your existing pictures to the iPad for review. The current version can do this; the new version will do the same, just redesigned to manage things internally and be much more crash resistant.

      We have some things in development now that will mean Lightroom may not be needed, but since they’re still being explored, I can’t say whether or not the new version will include them.

      At the end of the day, everyone’s workflow is a little different, and not every program (including Photosmith) is perfect for everyone. My recommendation is to start with the Camera Connection Kit if you use SD cards and/or follow this blog to see if it fits into your needs.

  11. Diane says:

    I don’t appear to suffer import crashes but on an import of say 100-150 images, I suffer two or three crashes and some significant lags on working theough the images in full screen mode (ipad3 and nikon d700).

    I only use this on a one time sync to do a first pass cull after a portrait shoot, so only have the last imported field ticked, and my camera roll number is around 50.

    I really want to be able to give my ipad to my clients to look through and star their favourites whilst i pack up, but i can’t yet. Will this underlying change improve the navigation through the photos?

    Thanks in advance.

    • Chris Horne says:

      In short, “yes”.

      The longer answer: Currently, Photosmith uses the Photos app when loading each and every image as you scroll through them. Every time we load a thumbnail, every time we load the full resolution image, etc. Each of these times that we access the Photos app, we run into possible stability problems. Partly because of “interesting design decisions” on Apple’s part, and partly due to Photos.app instability that we have no control over. Access the photos app enough and Photos app starts acting up, eventually causing us to crash.

      In the new version, the original photos can either be imported directly into Photosmith and previews created upon import, or the original photos can be linked to Photos.app, and the previews still created in Photosmith. In either case, the previews used for thumbnails and fullscreen are all under our control, leading to a much more stable and predictable system. Having everything under our control means that we can actually do something about the crashes rather than being under the mercy of another app or software library.

  12. Leo S says:

    I’m new to photosmith and i love its concepts. I’ve been wrestling with it for more than a week now, it has loaded an album with 4200+ photos without much problem (a couple of crashes near the end or a ~7hrs sync i think, but i didnt see it first hand), but i’m suffering from consistent crashes just from browsing and rating the pictures! Every 5-7 pictures it seems, then the app would hang (does not respond to gestures or taps at all) and if i switch to homescreen or another app, the last photo that was displayed by photosmith stays on screen, blocking my homescreen! (Ie: if i press the home button twice and swipe left (where ipad’s search function should be) then the keyboard will appear ofer the frozen photosmith ghost image. If i kill that photosmith instance (from the task selection bar) then the homescreen (or ipad search screen in the examplemabove) will appear.

    Interestingly, it seems that the photosmith instance may still be working in the background when the display froze. If i swipe a few times after it froze, it may display a different picture after i kill and restart it.

    Help, i really want to use photosmith in my workflow, but this thing is killing me! (And itself)

    • Chris Horne says:

      Leo-

      That’s one of the problems that will be taken care of with the new version.

      We had a lot of feedback when designing version 2, and something we heard many times was “I only need to sync 100 or so pictures at a time.” We focused our efforts on that scale, only to find out later that users such as yourself were syncing thousands (up to 20,000 from one user) of images. The internal caches we use simply aren’t built for that kind of load.

      And… unfortunately, those internal caches are required when using the Photos.app library. It’s simply the nature of how Apple’s library is written; we’ve tried a number of bandaids, and it’s obvious that we just can’t work within the, ahem, “constraints” of the Photos.app library. And thus this is why we’re redesigning.

      Our current target is to allow unlimited images, although practicality limits us to testing ~30,000 images.

      As for your current problem, my best suggestion for the time being is to limit your catalog to 1000 images or less. The app is running out of memory and crashing on you. If you still have problems, please contact the support guys from the Support page.

  13. Scott says:

    Will this new upgrade bring the long promised support for the seagate satellite hard drive? This would go a long way to allowing a lot of us to off-load our photos when traveling, etc.

    • Chris Horne says:

      The GoFlex drive and it’s current firmware won’t work for an on-the-go storage device, due to some technical reasons on Seagate’s end. We’re under strict NDA with Seagate, so we can’t comment on what those reasons are or what changes may be happening; however, I can tell you that progress is being made.

      I wish we could say more. Seagate has been very good to work with, and this is something high on my personal wish list, so we’re still doing everything we can to make it a possibility.

  14. [...] mit Lightroom bemängeln. Dessen sind sich die Entwickler durchaus bewusst und überarbeiten den vorhanden Import momentan. Eine Übersicht von getesteten Kameras findet ihr übrigens hier. Photosmith Hersteller: C [...]