One feature I’d like to add to Snapthread is something akin to the “For You” section of Photos: a small number of auto-generated movies that the user might find useful or meaningful. In Photos, these movies are based on common image properties such as date, location, relationships gleaned from facial recognition and contacts data, and image content classified via machine learning such as dogs, cats, and bodies of water.
I don’t have access to the facial recognition data that Photos collects, and as anyone who’s had the pleasure of syncing their iCloud Photo Library to a new device knows, feeding tens of thousands of photos through an image classification algorithm takes a long time and can burn processing and battery power like nobody’s business. That leaves me with two methods for grouping photos and videos: date and location.
Attempting to smartly group a user’s photos by location without knowing where they consider “home” is pretty much impossible. Are those vacation photos, or just bathroom selfies? I don’t know. I don’t want to know. That leaves me with the safest, least creepy option: capture date.
At the surface level, organizing photos for a user based on their date seems super innocuous—that is, until we stop to recall what a disaster it was for Facebook when they first presented auto-generated “Year in Review” videos to every single user. While some people smiled at memories of a happy year, others were intensely and abruptly reminded of lost loved ones, breakups, illnesses, and other emotional events. In fact, it was re-reading Eric Meyer’s heartbreaking blog post about it that made me pause and think twice about adding this feature.
Some years just aren’t good years. There’s no way for an algorithm to know that. There are, however, steps I can take as a developer to design my “For You” feature with the worst case scenarios in mind:
Make the feature opt-in. This would involve unobtrusively asking the user if they’d like to see some suggested movie projects. The preference would also be available as a toggle in the app’s settings.
Don’t auto-play anything. Even if a user has opted in, they may not want to view a particular suggested movie for whatever reason. I don’t want to force it on them.
Make the whole “For You” section collapsible. Maybe a user just doesn’t like that particular day’s suggestions. Let them hide the thumbnails so they don’t have to look at them.
Make the movies editable. Maybe there’s just one or two videos that ruin an otherwise great movie. Let users delete/replace them.
Don’t add any titles or captions that suggest a particular mood, like “Summer Fun” or “My Great Year” etc. Just stick to dates.
There are two types of auto-generated movies I’d like to offer: ones based on recent photos/videos (such as “last month” or “today”) that are designed to get users up and running quickly, and memories from awhile ago, such as “on this day.” I don’t think the recent ones need safeguards: after all, those are photos you’d see if you opened up your library anyway. It’s the ones from years ago that I need to be sensitive about.
Curating someone’s personal memories is challenging. At best, it can surprise and delight; at worst, it can be painful, invasive, and just downright creepy. We app devs may not have to take any sort of Hippocratic oath, but we probably should. If, like me, you’re considering adding some kind of personalized content to your app, tread carefully, and design with worst case scenarios in mind.
Yesterday I finally had some time to sit and think about what improvements I want to make to Snapthread this summer. I still want to rewrite the app using SwiftUI; however, after a bit of exploration, I think I may need to wait until it’s a little more capable. Here’s what I’m planning to do instead.
I want to leave the app in a good place for my iOS 11 and 12 users. To do that, I want to add a few more soundtracks to choose from and a tool for adjusting a video clip’s speed.
Based on everything that was revealed at WWDC, here’s what I want to do after I set the minimum OS target to iOS 13:
Rewrite my UICollectionView code to use the new compositional layout class and diffable data source
Redesign my photo library picker. Apple has deprecated the methods I was using to fetch “Moments,” so I will need to do something else to help users find the photos and videos they’re looking for.
Explore some of the new property wrappers, like @UserDefault
Replace my icons with SF Symbols and design a few custom ones
Replace my colors and font sizes with semantic ones and set the whole app to use dark mode
Use the new system provided font picker
Possibly rewrite two view controllers in SwiftUI: Settings and Soundtracks
If I have time, create some more custom Core Image filters
Doing everything on that list should help rid my code of most of its remaining bugs and set the app up well for the future. I can’t wait to get started!
Wow, what a conference, eh? Like most, I’m still processing the many new frameworks and APIs that Apple presented to us last week. So far I’ve watched 12 session videos, taken copious amounts of notes, and spent lots of time thinking about what all of this could mean for my app. As such, this post will be an attempt to organize those thoughts.
When I wished for more standard system icons that could be used anywhere, I definitely did not expect Apple to deliver over 1500 of them. I feel particularly validated by Apple’s instructions for creating custom icons: find a symbol in SF Symbols that resembles what you’re looking for and edit the SVG. I feel validated because that’s exactly what I’ve been doing to create all my icons in Snapthread, except that my custom icons are based on a $25 set of 200 icons from Glyphish. Browsing through SF Symbols, I think I can replace nearly all of my icons with them, with maybe two exceptions.
The Big Functionality Giveaway
One huge point that nearly every presenter hammered on was that if you follow the Human Interface Guidelines and use Apple’s frameworks as-is, you get a TON of functionality for free. In fact, one major goal of SwiftUI is to handle all of the basic features of your app for you, so you can focus on perfecting your app’s cool, custom features. For example, if you use SwiftUI correctly, the system will automatically handle animating view changes beautifully. If you use semantic colors, Dark Mode just works. Localization behaviors for right-to-left languages, Dynamic Type—these are all things you get for free if you use Apple’s semantic font sizes and SF Symbols.
I think it was Mike Stern who said something like, “if you spent time recreating what UIKit gives you for free, with custom controls, you may want to…I don’t know how else to say this…stop doing that.” Launch Storyboards, resizable interfaces, and support for split view multitasking will all be requirements starting in April of 2020. I don’t think the message has ever been clearer: follow the HIG, use the tools we’ve given you, be a good platform citizen. Just do it.
The New Peek & Pop
If you haven’t watched “What’s New in iOS Design,” you should. Peek and pop have become “contextual menus” that are now available and accessible on all devices. “Use them everywhere!” Mike says in the session. Apple wants these contextual menus to be so pervasive that their users expect to find them all over the place. An important thing to note is that any functionality placed into a contextual menu should also be accessible from elsewhere in the app. There are convenience methods for adding contextual menus to table view and collection view items, which I plan to use so that users can perform common actions on video clips in their timeline. Overall, I think this is a great change.
I wasn’t particularly excited about dark mode prior to the conference because my app, like most other video and photo editors, already has a dark appearance. However, now that I’ve learned more about it, I really like the way Apple’s colors, fonts, and new “materials” adjust to trait changes. For instance, if you use semantic background colors, there are slight variations for “base” and “elevated” states. Apps are elevated when in split view multitasking so that the black separator between apps can be seen more clearly, and controllers and views are considered elevated when they are presented modally. The whole system seems well thought-out, and I plan to adjust my code to use semantic background and font colors, as well as the new “materials” options, and then simply force the whole app to use dark mode (which, incidentally, is as easy as changing an Info.plist value).
I…don’t understand Combine yet. I mean, I sort of do. I don’t feel like I need to understand it yet, though, because there are only a few places in Snapthread where I could make use of it. I observe values on my AVPlayerItems, there’s a few UserDefaults I keep track of, and maybe a handful of Notifications. Anyway, I’m sure it’s really awesome; I just need to re-watch the videos and read a few more articles before I can grok it.
Collection View Improvements
Collection views got a major API upgrade this year with completely new ways to lay them out and configure their data sources. Like SwiftUI, the new layout API is both compositional, and declarative. The most common crash in Snapthread has to do with the collection view inside my custom photo/video picker, and I still haven’t managed to figure out what’s causing it. This probably sounds terrible, but: I’m hoping that by using these new APIs, the problem might just go away!
In fact, I’m hoping a whole pile of layout-related bugs will be eliminated, which brings me to…
My code sucks. It just does. I’m inexperienced, I’ve had no mentors or code reviews (by choice—I’ve had offers from many great people!), and there are fundamental concepts of programming that I only have a tenuous grasp of, at best. Despite my best efforts, I’ve utterly failed at using the MVC model. My views are all up in my model’s business, I probably have delegates where I don’t need them, or, on the flip side, other weird hacky ways of communicating between view controllers (like via viewWillDisappear and unwind segues, and all sorts of odd places) where I should have just used a delegate.
With SwiftUI, I feel like I can finally just burn it all to the ground. SwiftUI makes sense to me because it is declarative, and I love it because it forces its views to rely on a single source of truth. One of the items on my wishlist was for “every visual customization that is possible for a UI component [to] be editable in Interface Builder.” As a modern replacement for Interface Builder, SwiftUI delivers on this request with gusto. There’s a TON of advanced drawing stuff you can do with SwiftUI and all of it is immediately preview-able without building and running the app. That blows my mind!
SwiftUI has some missing pieces. There’s no control that provides the functionality of a collection view. You could probably hack together some HStacks and VStacks, but you wouldn’t get caching or cell reuse. For now, UICollectionViews can be wrapped in a UIViewRepresentable-conforming object to be integrated into SwiftUI. If you’re working with videos, you still have to work with AVPlayerLayers. Live Photos are still previewed in a PHLivePhotoView. I’m sure there are many other frameworks that make use of UIKit classes as well.
Still, my urge to re-write Snapthread is strong. By re-writing most of the app to use SwiftUI, I’m confident that I’ll be able to edit it on the go next year, when a first party, Xcode-like code editor will likely arrive on iPad. I’m also confident that it’ll be way less buggy, and way easier for future me to understand, since all dependencies will be so clearly defined. I’ll try to share some of my new SwiftUI knowledge as I go!
I’ll have to drop support for iOS 11 and 12. Before I do that, I want to add one more feature and maybe some more music to the soundtracks list. It’s going to be a busy summer!
I’ve been watching Tuesday’s WWDC session videos on my iPad Pro via the WWDC app, with Notes open in split view so I can scribble thoughts and information with the Apple Pencil. It’s been a really enjoyable experience!
Ok, my baby is asleep, my husband is napping with our toddler: let’s do this. tvOS! Multi-user support! (multi-user all the things please!!) Xbox and PlayStation controllers?! This is already nuts. #WWDC19
A few years ago around WWDC time I made the mistake of installing the fresh new beta of OS X on my only Mac. Shortly after that, I needed to submit an update to one of my apps…only to find out that you can’t submit release builds to App Store Connect (then iTunes Connect) from a beta version of Mac OS. After some furious googling, disabling system integrity protection, and editing some plist that I was undoubtedly not supposed to touch, I tricked Xcode into thinking I was using the previous version of the OS. Lesson learned.
Since then, I’ve waited until September to update my Mac. This year, however, is different. This year is Marzipan.
So, I took to Twitter and asked for recommendations for external SSDs. Several people recommended the Samsung T5 Portable SSD, so that’s what I got. Fortunately it arrived today, just in time to install macOS 10.15 on Monday!
If, like me, you’ve never run macOS from an external drive, I found some very good instructions over at Macworld. I’m looking forward to exploring all the new features (and Marzipan apps!) that macOS 10.15 will bring without worrying about messing up my main development environment. How about you? Will you be installing the new macOS beta next week?