Apple’s “Fuck You” to Mac Game Developers & Players

This wasn’t in the keynote, but Apple had some bad news buried in the “What’s New in macOS” section of their developer site for anyone who makes games or other software with the OpenGL graphics API :

Deprecations and Removed APIs
Periodically, Apple adds deprecation macros to APIs to indicate that those APIs should no longer be used in active development. When a deprecation occurs, it’s not an immediate end of life for the specified API. Instead, it is the beginning of a grace period for transitioning from that API and to newer and more modern replacements. Deprecated APIs typically remain present and usable in the system for a reasonable time past the release in which they were deprecated. However, active development on them ceases, and the APIs receive only minor changes to accommodate security patches or to fix other critical bugs. Deprecated APIs may be removed entirely from a future version of the operating system.

As a developer, avoid using deprecated APIs in your code as soon as possible. At a minimum, new code you write should never use deprecated APIs. And if your existing code uses deprecated APIs, update that code as soon as possible.

Deprecation of OpenGL and OpenCL
Apps built using OpenGL and OpenCL will continue to run in macOS 10.14, but these legacy technologies are deprecated in macOS 10.14. Games and graphics-intensive apps that use OpenGL should now adopt Metal. Similarly, apps that use OpenCL for computational tasks should now adopt Metal and Metal Performance Shaders.

Metal is designed from the ground up to provide the best access to the modern GPUs on iOS, macOS, and tvOS devices. Metal avoids the overhead inherent in legacy technologies and exposes the latest graphics processing functionality. Unified support for graphics and compute in Metal lets your apps efficiently utilize the latest rendering techniques. For information about developing apps and games using Metal, see the developer documentation for Metal, Metal Performance Shaders, and MetalKit. For information about migrating OpenGL code to Metal, see Mixing Metal and OpenGL Rendering in a View.

Apple is already requiring that apps get updated to be 64-bit, or they’ll stop working in a future update.

As much as I loathe John Carmack today, and it certainly didn’t help that he decided to write this on Facebook, he recently wrote about how he persuaded Steve Jobs to support OpenGL on the Mac:

I was brought in to talk about the needs of games in general, but I made it my mission to get Apple to adopt OpenGL as their 3D graphics API. I had a lot of arguments with Steve.

Part of his method, at least with me, was to deride contemporary options and dare me to tell him differently. They might be pragmatic, but couldn’t actually be good. “I have Pixar. We will make something [an API] that is actually good.”

It was often frustrating, because he could talk, with complete confidence, about things he was just plain wrong about, like the price of memory for video cards and the amount of system bandwidth exploitable by the AltiVec extensions.

But when I knew what I was talking about, I would stand my ground against anyone.

When Steve did make up his mind, he was decisive about it. Dictates were made, companies were acquired, keynotes were scheduled, and the reality distortion field kicked in, making everything else that was previously considered into obviously terrible ideas.

I consider this one of the biggest indirect impacts on the industry that I have had. OpenGL never seriously threatened D3D on PC, but it was critical at Apple, and that meant that it remained enough of a going concern to be the clear choice when mobile devices started getting GPUs. While long in the tooth now, it was so much better than what we would have gotten if half a dozen SoC vendors rolled their own API back at the dawn of the mobile age.

While OpenGL isn’t going away immediately in macOS Mojave, when it is finally gone there will be many fewer games on macOS, it has been the only portable graphics API available for developers to bring their games to Linux and macOS, as well as other platforms, for decades.

Without OpenGL on macOS the Mac and Linux will both suffer, as will new platforms. They’ll have a harder time getting games and other software when bigger platforms are locked to vendor-specific APIs like Metal instead of cross-platform ones like Vulkan and OpenGL.

If I had to guess, I would hope that Valve will ship an intermediary layer to translate OpenGL calls for games on Steam, and hopefully they will make this software available for everyone else. There are already some other projects to translate OpenGL to platform-specific calls but it’s not going to be easy for games to support them. It’d be better if these projects had something to handle the translation on-the-fly. It’s also entirely possible that Valve will just give up on older games supporting modern versions of macOS after Apple fully deprecates OpenGL.

I don’t envy anyone trying to support old software and write good OpenGL drivers like Apple has (even when they don’t update their OpenGL support for years), but the deprecation of OpenGL is a real “Fuck You” to game developers and players unlike any other. Games getting updated from 32-bit to 64-bit, as well as going through the process of having any kind of graphics portability layer added on top, seems unlikely. Thousands of games are going to be lost to time when OpenGL dies off. Competition with popular hardware and software platforms will be even more difficult. I understand the desire to get rid of technical debt, but this is bad.

Apple’s WWDC 2018 Event Notes

Here’s everything that Apple announced today.

Keynote

Here’s everything that Apple announced at their World Wide Developer’s Conference Keynote for 2018. The event opened with this four-minute nature documentary style advertisement on migrating developers:

iOS 12
Craig Federighi boasted about iOS 11 being available on iOS devices from 2013, and that the majority of their users are on 11 versus the 6% they claim are on the latest Android update. According to Google, the percentage of Android users running their latest operating system (Android 8.1) is 0.8%, ouch. I would guess that Apple based their estimate on the percentage of devices that are capable of running 8.0 or 8.1, which is 4.9% and 0.8% respectively. The majority of Android users (25.5%) appear to still be in the stone age on Android 6.0.

According to Federighi the primary focus of iOS 12 will be performance, specifically he said that older devices would perform better. Using an iPhone 6 Plus as an example, Federighi claimed that apps would launch 40% faster, the keyboard would display 50% faster, the camera would open 70% faster.

USDZ
Because this keynote is ostensibly for developers, Federighi announced a new format for 3D scenes to be bundled together, USDZ. Pixar created the USD format in 2012, and this is their update to it for 2018 (thanks to coolpowers).

Adobe’s CTO Abhay Parasnis announced that their products would support the new format and that Apple’s developer State of the Union would have some kind of demo of this integration.

Measure
Federighi announced a new app for augmented reality, Measure, which will presumably replace everyone’s first ARKit app by giving you measurements for real-world objects. The app can detect some shapes and automatically provide measurements for those, but you’ll need to tap at the ends or corners of most real-world objects to get the app to measure them.

Apple’s USDZ Demos
Federighi showed off a USDZ 3D scene of a fish embedded in their Apple News app, and then customized a guitar on Fender’s web store only to display it on the stage through an iPad’s AR viewfinder.

LegoARKit 2
Federighi said that iOS 12 would include a new version of their ARKit API for developers to create augmented reality experiences. The flagship feature for ARKit 2, shared experiences, allows multiple devices to view and interact with the same scene. A slide depicted a pre-recorded video of people playing a multiplayer AR game where both participants and an observer could see and interact.

Martin Sanders, the Director of Innovation at Lego, demonstrated the new object detection features of ARKit along with shared experiences by scanning a physical Lego set and then awkwardly holding two iPads around it with someone else from the Lego company. The iPads showed a city being built around the town square Lego set with some limited interactivity to place vehicles and minifigures. It looked like it could be fun, but this kind of AR scanning will probably be limited to sets that Lego produces, not the modifications people make to them. Sanders said that the Lego AR experience app would ship later this year.

Search in photos

Photos Search & For You
Federighi said that Photos would improve in iOS 12. He highlighted the search functionality that would offer suggestions and other improvements to help you find photos you’re looking for. The app will also get a new tab, For You, that offers suggestions of photos you might like to see or ways you could change them to make them more appealing. It seemed like the editing suggestions were limited to ones that involve Apple’s features like looping a live photo or making some specific change to a portrait photo.

The For You tab will also suggest sharing photos with people who appear in them. He said that they would be shared at full resolution, which is an improvement because photos shared today aren’t at the original resolution. Federighi also said that the recipient of shared photos would be prompted to share photos from the same event so that everyone ends up with the entire set.

Siri Shortcuts
Federighi mentioned a new feature called shortcuts that would let Siri easily start functions of other apps. The first example was “I lost my keys” which would have Siri open part of the Tile app’s functionality within Siri and look for them. Apps in iOS 12 would make suggestions of shortcuts to add to Siri. The iOS 12 lock screen and search will make suggestions of things to do like re-ordering coffee or turning on Do Not Disturb when you’re at a movie theater.

Federighi said that you’d be able to make your own shortcuts in a new “Shortcuts Editor” app.

Shortcuts editor

Apple’s Kim Beverett demonstrated the shortcuts functionality with a few examples. One of which was adjusting a shortcut she made in the Shortcut Editor. Beverett’s Heading Home shortcut would compress a bunch of steps into one action. It lets her roommate know how long it’ll take her to get home, set her thermostat to 70 degrees and turns on a fan, and opens maps with the course home preconfigured. Beverett quickly used the editor to play a radio program she wanted to listen to every time she goes home whenever she uses the shortcut. This Shortcut Editor is clearly the Workflow app that Apple acquired last year.

Other App Updates
Apple’s Susan Prescott demonstrated a few other iOS 12 app updates. Apple News is going from a 5 tab interface to 3 tabs with a new tab called Browse to handle the functionality from the current Search and Following tabs.

The Stocks app is completely rebuilt, and has business news built-in and new charts. The app will also be on the iPad.

Voice Memos is “even easier to use” and is also on the iPad, it’ll also use iCloud for file synchronization.

iBooks is getting refurbished with a new name and interface, it’s just Apple Books on iOS 12.

CarPlay is going to support 3rd party navigation apps.

Dnd

Do Not Disturb
Federighi returned to show us how iOS’ Do Not Disturb will be updated with a new feature called Do Not Disturb During Bedtime. It will hide notifications so you won’t be “barraged” with them if you wake up in the middle of the night and just want to see what time it is. You can also granularly enable Do Not Disturb until you leave a location or for a certain amount of time.

Notification Updates & The Screen Time App
You’ll be able to change the way notifications are delivered, or turn them off entirely, directly from the lock screen. You’ll also get suggestions to disable them for specific apps when you rarely interact with them.

Notifications will also be grouped together by what app sent them.

The Screen Time app will give you a weekly report of how you’re wasting your life on your devices. It’ll also let you set time limits for each app. You’ll get reminders about that limit when you’re about to reach it, or you can set limits for the apps your kids use by category or by picking each specific app.

 

Animoji

Animoji & MeMoji
Animoji are getting updated to let you stick out your tongue with your 3D avatar, they’re also getting four new characters. The ghost, koala, tiger, and t-rex. You can also make a character based on your own face, Apple calls that a “MeMoji.”

Kelsey Peterson demonstrated these effects on stage, and the selfie camera can now display the characters over your real face.

Facetime
Facetime can now have up to 32 people on a group video chat. It can be launched directly from a group text message chat in Messages. The different camera effects, like Animoji, are also available on Facetime.

WatchOS 5
Apple’s Kevin Lynch talked about new improvements for fitness and communications.

WatchOS 5 is getting improvements to compete with your friends in exercise. The Workouts app is getting a Yoga workout type, hiking, running gets a rolling pace measurement, pace alerts, and cadence measurements. WatchOS 5 will also automatically detect when you’ve started some types of workouts, it’ll retroactively credit you with the workout if you tell the Apple Watch to start the workout late.

There’s a Walkie-Talkie app for WatchOS 5 that goes over cellular or wifi connections.

The Siri watch face is getting information about sports, map information, and your heart rate. Siri Shortcuts will be on that Watch Face, as well as third-party apps.

Raising your wrist will let you talk to Siri right away, you won’t need to use the “Hey Siri” hotword.

Watch notifications

Notifications can also be more interactive, like on the phone. One example was getting a notification from Yelp about a dinner reservation

Apple’s WebKit embedded browser will be available for some web links on the Apple Watch.

The Podcasts app is going to work on the watch, it’ll sync full episodes from your phone.

Apple’s Julz demonstrated these new features while riding an exercise bike, which was pretty impressive.

Kevin Lynch returned to tell us about Student ID Cards coming to the watch, it’ll work with just 6 universities to start, but it includes Temple University in Philadelphia.

Lynch also introduced a new rainbow watch band and watch face for Pride month. The watch face is supposed to be available today.

Apple TV 4K
Apple’s Jen Folse talked about how the Apple TV 4K from last year is getting Dolby Atmos in the next version of tvOS, she also talked about how movies will get free upgrades to support the new audio format.

Folse reiterated Apple’s support for live TV and sports.

Folse also said that various cable providers are letting their users replace their cable boxes with the Apple TV, and use a new “Zero Sign-On” instead of the single sign-on system from last year.

The Aerial screensaver on the Apple TV is going to tell you what you’re looking at, finally. It’ll also get a new view from the International Space Station of the Earth.

Mojave

macOS Mojave
Craig Federighi returned to talk about the new features in macOS Mojave.

As Apple leaked, the next version of macOS is getting a dark mode. Previously you could dim the menu bar dock in their settings, but this is a system-wide feature that is more complete.

Desktop Stacks is another new feature that organizes the crap that gets scattered all over your desktop into stacks of documents matched by type, or date, or by tags. It’ll automatically keep those stacks organized as new files are added to the desktop.

Finder gets a new Gallery View that appears to replace the Cover Flow view with thumbnails at the bottom of the screen and a quick look type of view at the top.

The Finder is also getting a detail side bar with shortcuts to different actions you might like to take on the file, Federighi specifically mentioned adding customized automator actions to the side bar for certain types of files like watermarking files with one that was put together earlier.

The quick look utility is enhanced to let you do more from it, Federighi demonstrated trimming a video without opening a separate editor and signing a PDF document.

Screenshots are enhanced to be similar to the iOS functionality but with more powerful functionality that you’d expect on a computer. You can also capture video directly from that utility now.

The Continuity Camera feature was demonstrated by Federighi to take a photo from his phone and pop it directly into a Keynote slide. It can also scan in documents from your phone directly into a Mac app.

A few iOS apps are hitting the Mac. Apple News, Stocks, Voice Memos, and Home for using HomeKit devices.

Mac voice memos

Privacy
Mojave will have better privacy enhancements to prevent other apps from getting access to your personal data unless you want them to do so.

Safari on Mojave and iOS 12 will prevent advertising publishers from tracking you based on sharing functions and comment fields. Mojave and iOS 12 will also hide more of your information from being fingerprinted by websites and advertising publishers.

New mac app store

Mac App Store
The Mac App Store is finally getting updated, it hasn’t had many changes since 2011. Apple’s Ann Thai showed off a redesigned UI that takes a lot from their iOS App Store. It looks much better.

Metal & Core ML on iOS and macOS
Federighi returned to talk about Apple’s Metal graphics and computation API and external GPUs. He boasted about the speedups that Macs can get from up to four GPUs. It’s kind of crazy that the only GPUs that Apple supports for external GPUs are from AMD.

Apple’s machine learning API, Core ML, is getting updated with a 2nd version. It’s supposed to be 30% faster with a 75% reduction in the model size.

UIKit on the Mac
As has been rumored, Apple is making it easier to bring iOS apps to macOS. No specific name was given for this technology, but Federighi says they’re using it internally with apps like Apple News and promises that it’ll be available for 3rd-party developers next year.

Release Dates for iOS 12 and macOS Mojave
“This fall”

Overall
I’m very happy that macOS is getting updated with a new Mac App Store, it’s been way too long and developers have been abandoning it for independent distribution systems. I like having apps bundled into one store, but it’s also good that Apple finally wants to compete with independent distribution.

It’s a little disappointing that the iPad Pro didn’t get updated with this event, but perhaps they’re moving the majority of hardware updates to the Fall event.

I’m very happy to see that Workflow is still getting updated, and it looks like it is even more useful after Apple has updated it under the Siri banner. I wish that it were on the Mac as an alternative to Automator.

Homebrew Users: Run brew cleanup & Get Your Gigs Back

If you use a macOS machine for development, or even just to get some bonus commands that you wish the system came with, then you probably use Homebrew as your command-line package manager.

Perhaps you’re like me, you’ve been using it for years and didn’t realize that it has left gigabytes of detritus on your local drive, I certainly didn’t until someone in the homebrew IRC channel mentioned the brew cleanup command, and now I have 21.3 gigs of disk back.

Joakim Sandberg’s Iconoclasts Out on Steam, PS4, Vita

It can be pretty frustrating to find out that something you want to fix is difficult or impossible to repair. Glued-on screens cover batteries that are all custom fit inside small cases that prevent curious people from learning how things work and fixing problems with their devices. Iconoclasts from Joakim Sandberg takes that a step further, it’s a world where a mechanic, Robin, finds that her profession is outlawed. Your mission is to get Robin and her friends together to fix things in what looks like a bit of a metroidvania side-scrolling action-adventure with a Metal Slug-y vibe to the art.

Andy Kelly likes it:

Iconoclasts is a fine game, offering both satisfyingly sharp platforming and shooting, and some really smart puzzles. It’s enormous too, packed with secret areas and other stuff to discover. And although I found the humour a little glib and childish at times, it tells its heartfelt story well. A lot of Metroidvania games go for a bleak, downbeat atmosphere, but Iconoclasts is infectiously vibrant and sunny, even if the story does occasionally venture into dark territory.

Iconoclasts is out now for $20 on Steam on Windows, macOS, and Linux, gog (same platforms)as well as the PlayStation 4 and Vita.

Apple Has a Patch out for the macOS Root Access Security Vulnerability

Go to the Updates tab in the Mac App Store to apply it now, you won’t even need to reboot. Apple has more details about the update at this link.

Here’s the post from yesterday with the details of the vulnerability.

Update:
If you have any trouble with file sharing after applying this security patch Apple has another fix for that, oops.

The macOS Root Access Security Vulnerability

There’s a vulnerability in the latest version of macOS High Sierra (10.13.1) that may let anyone with physical access to a Mac log in and gain system administrator (root) access. Or, if they already have an account, upgrade their access to the system administrator (root) level.

You can work around the issue by setting a root password as described in this support document from Apple. They’re working on fixing it.

The vulnerability works like this:

  1. At any login or a privilege escalation dialog a user types in the username root
  2. The user hits the login button or enter a few times in quick succession
  3. The system enables the root user account and assigns it no password.

This is incredibly bad for Apple to have a vulnerability this easy to exploit, and it’s ridiculous that it was also apparently publicly available on Apple’s developer forums weeks ago.

Apple’s WWDC 2017 Press Event Notes

Every year Apple has at least two big events, their iPhone event in the second half of the year and their developers conference event in the first half of the year. Each event has it’s own press briefing, here are my notes from today’s WWDC 2017 press briefing. It is long as hell.

The show opened with a comedy skit about a new Apple engineer sitting down at a new desk and accidentally unplugging the iOS app store servers, disabling every app. Soon, the world descends into chaos as people can’t get directions and we see a marketplace of people trying to substitute for the missing apps:

Tim Cook was the first live human on stage with numbers and then to announce that Amazon’s Prime Video service is finally going to the Apple TV.

WatchOS 4:
Kevin Lynch announced new Siri functionality that would automatically present information you might want on the new Siri watch face. Rotating the crown scrolls through the available information. For example you might want to see your upcoming appointments and weather changes. They’re displayed as tiny cards as seen here:

There’s also a new kaleidoscope face along with several animated Toy Story character faces.

WatchOS 4 also has more customized notifications for activity encouragement and better reward animations when you’re done exercising.

The exercise tracking is more advanced and detects when you’re paused while swimming to determine your intervals, for example. It’s also easier now to move from one workout type into another without stopping.

New gym equipment will be Apple Watch enabled and will help you record your fitness activities on that equipment when you swipe your watch near a certain area on the equipment.

Apple Music is improved on the watch with a new app.

The new WatchOS will be out this Fall with a pre-release version available to developers today.


Mac Updates:
Tim Cook introduced Craig Federighi to talk about macOS updates. macOS 10.13 will be called High Sierra, another stability update along the lines of Snow Leopard and Mountain Lion. Craig joked about the name’s obvious weed reference.

Craig promises these new features in 10.13:

Faster Safari web browsing with faster Javascript.

Safari will also block autoplaying video and audio on web pages. That’s great.

Safari will have “Intelligent Tracking Prevention” to prevent sites from identifying you across domains.

Search is better somehow. Apple didn’t really explain how

Mail gets a split view in full-screen mode to edit new messages alongside reading others and is also more optimized for storage.

Photos gets all kinds of improvements such as improvements in detecting faces.

Finally, Photos’ machine learning stuff gets synced across devices.

Photos also has way more advanced editing features on the Mac.

Apple’s Apple File System came to iOS recently to replace HFS+, APFS will be on High Sierra as well.

MacOS will also support h.265 for video compression, which will also be supported with dedicated hardware on Macs.

Craig boasted about their graphics API Metal for a while before announcing Metal 2 that is supposed to give a 10x draw call throughput improvement. It’ll also have better debugging and optimization tools.

Interestingly, Craig talked about how there is a new Thunderbolt 3 enclosure with an AMD Radeon RX 580 and USB-C hub available for developers who want to offload Metal GPU computation tasks to an external device from a Mac. It isn’t clear if this can offload desktop graphics rendering, as well, but you would assume so.

Craig announced VR support coming to the Mac from Valve with SteamVR and engines from Epic (Unreal) and Unity.

All systems that support Sierra will support High Sierra. It’ll be out this Fall with a public beta later in June.


Mac Hardware Updates
New iMacs are coming with 50% brighter displays and Kaby Lake chipsets from Intel. The iMacs are getting support for more RAM, 32 gigs on the 21.5 inch and 64 gigs on the 27. The 27 also gets fusion drives by default. The 21.5 iMac gets improved integrated graphics. The 4k 21.5 inch iMac gets the Radeon Pro 555 and 560 with up to 4 gigs of RAM.

The 27inch 5K iMac gets the Radeon Pro 570, 575, and 580 with up to 6 gigs of RAM.

MacBooks and MacBook Pros are also getting updated with new chipsets.

Industrial Light & Magic’s John Knoll & Epic’s Lauren Ridge showed up to give a VR demo running on the new iMacs. The demo featured live edits to a Star Wars scene in the Unreal Engine. It was incredibly awkward and didn’t demonstrate anything new over what Windows desktop machines could do last year.


iMac Pro

 

There’s a new 27 inch iMac Pro in Space Gray to offer an option for anyone missing the Mac Pro that hasn’t been updated in years. With copious options in CPU (up to 18 cores) and RAM (up to 128GB).

According to Apple it’ll be much cheaper than a comparable workstation that would cost over $7000, this one will start at $5000. It’ll be available this December.


iOS 11
Tim Cook boasted about iOS 10’s lack of fragmentation compared to Android and then reintroduced Craig Federighi to talk about iOS 11.

iMessage apps get an updated app drawer.

iMessages is now in the cloud to sync your message history to all of your devices, and it is still encrypted end-to-end, which should mean that Apple can’t access your messages in any situation. Although practically they could do so and you might not know about it.

Apple Pay is getting person-to-person payments, and it’ll have integration with Messages. Although it’ll store by default to an Apple Pay Wallet you can transfer money to your bank account. Sounds like the introduction of an Apple Bank.

Siri is getting an improved voice that sounds more natural and is still the only option with both male and female voices. Siri will also support translation in beta with some languages. SiriKit for developers will do more and support more applications. The last update only worked with a few custom actions like sending messages in third-party apps.

iPhone photos and videos are getting new compression formats that are supposedly much more efficient. The portrait camera is getting more features for a higher quality end-result.

Memories are enhanced. Video editing is supposed to be much improved as well.

Control Center is now modular and a single page. If you 3D touch a control module you get more controls and more information for that module as it expands to take up more space on the display.

The lock screen notification area will now expand with a swipe to display all of your notifications, not just the most recent ones.

Live Photos are improved, you can pick the key photo and loop the short video in live photos or choose other effects.

Maps gets improved maps for inside buildings like malls and airports. Navigation gets speed limits and lane-guidance. There’s a new feature for drivers called Do Not Disturb while Driving, it’ll be automatically suggested while driving and hide notifications. It can also notify anyone texting you that you’re driving and that you won’t see their messages unless they’re on a VIP list or they can break through by responding with “urgent.”

AirPlay 2 supports multi-room speaker systems from all iOS devices as well as the 4th generation Apple TV.

Apple Music can optionally share your playlists and music library with your friends, developers can also access this information through new APIs.

Phil Schiller appeared on stage to announce some updates coming for developers and users. The iOS App Store gets a complete redesign that looks more like Apple Music. A new Today screen shows off new cards with different stories about new apps. There are new dedicated screens for showing off games and a different screen for non-game apps. Category editors can display videos demonstrating tips and tricks for different apps.

Craig came back to talk about other technology improvements, including ARKit for augmented reality. It’ll work with Unity, Unreal Engine, and Apple’s SceneKit. Craig demonstrated 3D objects like coffee cups and lamps displayed on surfaces like tables that are detected instead of just having the 3D objects overlaid on what the camera sees without any interaction with real-world objects. An updated version of Pokemon Go will support this new ARKit and display Pokemon on the ground instead of just floating in the air.

Peter Jackson’s WingnutAR team appeared on stage to demonstrate an AR application running on an iPad with a science fiction action scene. Using AR to display a pre-configured action sequence is pretty boring, honestly.


The 10.5 inch iPad Pro

Apple introduced a new iPad Pro with a 10.5 inch display and an A10x processor. Joz told us that this supports a full-size software keyboard along with a better keyboard case. It’ll have a better refresh rate that looks smoother when motion is displayed on the screen, so Apple called this ProMotion. It’ll also dynamically adjust the display refresh rate for whatever the content is you’re working on, for better performance or battery life at a slower refresh rate.

The new 10.5 inch iPad Pro also has the same camera as the iPhone 7 and the base model has 64 gigs of storage. They’re available for order today and will ship next week.

Craig came back to talk about iOS improvements for the iPad. The dock can now have a large amount of apps. It can be pulled up from the bottom of the screen in any app. You can pull apps out of the dock into a slide-over multi-tasking split-view. iOS 11 will also support drag and drop. There’s a new Files app that supports iCloud drive and third-party cloud file management services like Dropbox.

The Notes app now supports handwritten notes that get interpreted using OCR as text by the system for searching and can scan a document for editing.

The 12.9 (let’s be honest, 13) inch iPad Pro also gets the new display, processor, increased base storage (64GB) and other features. All of these iPads 10.5 and 12.9 also have the 2nd generation TouchID sensor with quicker fingerprint recognition, but still appear to have physical home buttons instead of the haptic-driven virtual home buttons of the iPhones 7 and 7 Plus.


HomePod

Phil Schiller came back on stage to talk about “Reinventing Home Music” with a new speaker from Apple called HomePod. The first non-Beats branded speaker from Apple since the iPod Hi-Fi. HomePod is a rotund  7-inch tall device with a little bit of an urn shape and has its own A8 chip inside. Supposedly this speaker is very advanced and detects the kind of environment it is in to adjust for different types of rooms and adjust the audio for clarity. It’ll also work with another HomePod automatically for better stereo separation.

Of course, this speaker will support Siri and be Apple’s competitor to Amazon’s Echo and Google’s Home device as was rumored for months. Apple says that this speaker will have advanced Siri functionality that is tailored specifically to it.

The HomePod will be $349, in white and space gray, this December.


Overall, this feels like Apple trying to catch up, in VR and AR, with a speaker for Siri and finally getting updates to the iPad Pro and an iMac Pro to help reassure Mac Pro users like video editors that Apple hasn’t completely abandoned them when the Mac Pro still doesn’t have a release scheduled beyond “not this year.”

It’s good to see VR on the Mac, but nothing today addressed the desire for a desktop Mac for gamers to play VR applications unless you want a machine with a built-in display and a GPU you can’t upgrade. As terrible as Nvidia can be as we’ve seen with the GTX 970 memory debacle, I don’t ever want to buy a desktop with an AMD graphics chipset.

I would probably want a HomePod as Apple is the only company offering an AI assistant speaker that gives a damn about privacy and isn’t just trying to sell you shit through their store like Amazon with their Echo.

iOS & macOS Updates

Today’s software updates from Apple are notable for a lot of reasons. macOS 10.12.4 gets Night Shift from iOS, I can now find my AirPods in the Find My iPhone app, there are the beginnings of a standard to control apps that nag you for reviews on iOS 10.3, Siri enhancements on watchOS 3.2, but the most important change is he first switch from HFS+ to APFS. APFS is Apple’s new file system for iOS, macOS, tvOS, and watchOS. It’ll be installed with the latest iOS update, and will come to macOS in the future.

File systems keep track of where your files are on a storage medium like a hard drive or SD card, and Apple has been using some variation of the same file system, HFS and HFS+, for the past thirty years.

APFS should be an improvement in performance and reliability because it is targeted towards modern computing devices that use flash memory instead of spinning disks, but it is going to be a little bit like pulling a table cloth out from under a plated dinner and table setting in order for the operating system to replace the file system on the devices during the update.

I made sure my iCloud backups were all up to date before starting the upgrade, you should too.

Apple’s WWDC 2016 Keynote Notes

Here’s what Apple announced at their WWDC 2016 Keynote, or you can just watch the video.

WatchOS 3

Kevin Lynch spoke about the changes coming in WatchOS 3.

Apps, that you pick, update in the background and launch faster as a result.

The long button below the crown will no-longer launch your VIP contact list. It’ll now launch the dock app switcher. The dock displays a live view of the applications so that you can see updated information even without entering one.

Glances are gone, replaced by control center when you swipe up on the watch face.

Messages immediately give you options to reply, instead of having to choose that you want to reply first.

When a message comes in you can scroll down to get a list of suggested replies.

WatchOS 3 will have a new input interface called Scribble. You draw out letters one at a time with the whole word appearing above the input interface. This supports both English and Chinese characters.

In addition to the Micky Mouse watch face you can now choose Minnie Mouse with different outfit color options.

There is a new watch face called Activity that more prominently features the activity rings which monitor your steps and other exercises throughout your day. They’ll be larger and displayed behind an analog style of watch hands if you choose this face, or in chronograph and digital variations.

Kevin Lynch isn’t done with new watch faces yet, Numerals is another. This one displays only analog watch hands and a prominent digit or digits in many different fonts for the current hour.

Another change to the watch face. You can now swipe to switch between them and choose which you would like available.

More complications are available on more watch faces.

Stacy Lysik gave a demo of WatchOS 3. She shows the audience how Apple’s watch apps have been updated for quicker interactions. The timer gives you a few preset timing options to launch, for example. This should allow my son to more quickly set timers that confuse me when they go off because he loves to grab my watch and mess with it while I am holding him.

Kevin Lynch returned to tell us about the SOS feature that can call your regional version of 911 and alert your emergency contacts with your location after the call. It’ll be activated in WatchOS 3 by holding down the side button and it’ll display a countdown before calling emergency services. Interestingly, Lynch mentioned that it’ll work either via bluetooth to your phone or over wifi if your Apple Watch is on a known network.

I wonder about the usefulness of an emergency call where your watch is broadcasting that call publicly on a speaker instead of through your phone’s earpiece.

Jay Blahnik appeared on stage to introduce a new fitness related improvements, starting with activity sharing as a competitive feature. With it you’ll be able to see your friends and family member’s activity rings and data like steps and calories burned. If your friends use third party apps and devices, it sounds like those will work with this feature as well as long as they use Apple’s HealthKit as a data intermediary.

When you view a friend’s activity information you’ll be able to send them suggested encouraging or competitive messages about their progress.

Jay Blahnik continued on to introduce new technology to recognize activity information for wheelchair users. He talked about how they have adjusted different notifications and other information if the wheelchair setting is enabled. For example, the Apple Watch can optionally remind you to stand for a minute once an hour. For wheelchair users the watch will remind you to take a break and push around a little. Some of the workouts will be specifically for wheelchair users.

Finally, Blahnik introduced an app called Breathe to remind people to take stress reducing breaks. It’ll have different kinds of optional reminders and will be able to guide you through a session with visuals or haptic feedback.

Lynch returned to the stage to discuss how WatchOS 3 will be improved for developers to enable integration with those new features and improvements.

Apps on the watch will be able to use Apple Pay. Fitness apps will be able to run in the background during workouts and have access to more data that will enable new kinds of workout apps on the watch according to Lynch.

There are many more improvements and new APIs to the SDK for the watch including SpriteKit and more. Games should be much better though it’ll still be a tough sell to keep your wrist raised up for a while. I’d still expect it to be really only for quicker interactions.

Eddie Cue came on stage to talk about tvOS for the 4th generation Apple TV. He spoke about new apps like Sling, Fox Sports Go, the French TV service Molotov and a few games like NBA 2k, Minecraft Story Mode, and Sketch Party.

Cue then introduced improvements to the Remote app for iPhone which has all of the features of the physical 4th generation Apple TV Remote.

Siri for the Apple TV will be able to find shows and movies by topic. Cue searched for high school comedy movies from the 80’s and got Ferris Bueller’s day off and other options.

Siri will be able to load into live channels. MLB was notably absent.

Logins with cable providers will be reduced to a single sign-on. The App Store will let you know what apps/channels you have access to once you sign in.

The Apple TV will have a dark mode, and will automatically download available apps if you download them to another device.

Cue finished the tvOS talk by briefly mentioning some of the improvements for the developer kit and mentioned that the new version will be released publicly this Fall.

Craig Federighi was introduced to discuss the improvements to OS X, now renamed to macOS. The new version will be macOS Sierra.

Sierra will introduce new features cribbed from third parties such as unlocking your Mac when you have your Apple Watch close to it.

Another new feature borrowed from a third party, Tapbot’s Pastebot in this case, is Universal Clipboard. You’ll be able to copy and paste text, images, video and more between Macs and iOS devices.

iCloud Drive will now include your files in the desktop folder and sync those between Macs as well as making them available via the iCloud Drive app on iOS.

If you are running out of space on your Mac’s hard drive, Sierra can attempt to offload older files to iCloud and remove other kinds of files that users don’t typically need like old application caches. There will be a new GUI specifically for doing all of these functions.

Apple Pay will now work on macOS Sierra through websites that support it. You’ll authenticate it on your iPhone with the Touch ID finger print reader or Apple Watch.

Craig announced another Sierra improvement, tabbing application windows like web browser tabs. Developers won’t need to do anything to support this functionality.

Videos will be able to go into a picture-in-picture mode from websites. You can drag to move or resize this always-on-top window which also persists across full screen application virtual desktops.

Finally, Siri will be available in macOS Sierra. She made a few jokes with Craig during a demo. Siri will be able to understand new questions that are more appropriate for a Mac. For example, Craig asked about files from a specific time period and location, and then did a follow up question to further refine the search. It is unusual for Siri to understand context.

Results from Siri can be pinned to the macOS Notification Center. Image results can be dragged directly into applications from Siri, or copied and pasted from an iOS device.

Sierra will be supported on these models of Mac:
Late 2009 & later
MacBook
iMac

2010 & later
MacBook Air
MacBook Pro
Mac mini
Mac Pro

Craig moved on to iOS 10 and discussed 10 new features.

First off the bat, user experience improvements.

The lock screen has been redesigned. You’ll be able to raise your phone and it’ll wake up, bypassing the issue with newer Touch ID sensors that are so fast you never see the lock screen if you press a finger to the home button.

Notifications on the lock screen are more interactive and designed for 3D Touch to display more contextual information. Craig demoed 3D touching a calendar invite and seeing more details about it, before accepting it.

The next example that Craig demonstrated was a more interactive iMessage conversation where he could see more of the context of the conversation and also get images in it without leaving the lock screen notification.

If you use the illegal taxi service, Uber, their updated notifications showed you the location of your incoming car with an unvetted driver that is probably making less than minimum wage after all is said and done.

Notification Center now lets you clear all of your notification with a 3D Touch gesture that reveals a clear all button.

Control Center was simplified in its initial display, but it also has another page you can swipe to with more bigger buttons for music control and an album art display.

It’ll be easier to get to the camera from the lock screen, you can now slide from right to left to open the camera application.

Sliding from left to right displays a new widget view. The demo broke when Craig tapped on his calendar widget to “show more” of his day. An engineer was immediately brought on stage to answer for this failure.

There are more 3D Touch improvements to apps on the home screen. The email application can now display a widgetized list of contacts and a count of unread messages from them. The activity application on the iPhone can also display your activity rings in a widget that appears when you 3D Touch on that from the home screen.

These widgets can also display live video through third party applications like ESPN.

Craig moved on to iOS 10’s improvements in Siri.

Developers will have access to Siri in IOS 10. Though it appears to be limited to certain categories of apps like messaging apps. Craig mentioned Slack, Whatsapp, and WeChat.

Siri will hail illegal taxi cabs through third party apps Uber, Lyft, and Didi. Search photos in Pinterest, IM and Shutterfly. Start and stop workouts with MapMyRun, Runtastic, and Runkeeper. Send payments with Number 26, Square Cash, and Alipay. Set up VoIP calls with Cisco Spark, Vonage, and Skype.

CarPlay will also work with third-party apps for messaging and VoIP.

The keyboard suggestions, QuickType, will be improved and understand the context of the conversation. Craig’s example, is that it’ll understand the difference between playing in the park versus the Orioles playing in the playoffs. If you’re asked “where are you?” in iMessage, QuickType suggestions will give you a big suggestion to give your current location on a map. If you’re asked for contact information, QuickType will suggest sending the contact. The calendar event suggestions that you’d see if you tapped on text like “Sunday at 2PM” before will now be more context-aware and understand that you’ve been talking about a certain type of food and a street address that get included in the calendar event suggestion.

The keyboard will now support multilingual typing, I won’t have to switch between German and English anymore!

Photos will now display your photos pinned on a map, and will run facial recognition to understand who is in what pictures. I hope it works better than iPhoto did at that. Photos will also now detect objects and scenes in pictures so that you can search for these better. Craig displayed an example picture of someone riding a horse by a lake with a mountain, and said it would understand the mountain and horse and the scenery if you search for those. I wonder if we’ll see progress bars scanning all of our photos the first time we open the new Photos app in iOS 10 as this occurs. I suspect that this will be done offline due to privacy concerns.

Photos will also group together different trips, people, groups, and topics like “on the water” or “at the beach” into a new interface. Craig demonstrated the new “Memories” tab in Photos on iOS 10 that shows the product of this computer vision work. It’ll even create a short video of different events with Ken Burns style zooms of different photos and videos as well as music layered on top. Craig assures us this will pick the right music, but he demonstrates overriding the music choice and how Photos will re-edit the “memory movie” to match the music change. These changes will also come to macOS Sierra.

Craig takes a break as Eddie Cue returns to show us updates to maps. iOS 10 Maps will offer you some suggestions based on calendars and where you commonly go at different times of day. If you search for restaurants it’ll display a horizontal list of cuisines and restaurant styles you can pick from, and adjust its suggestions based on which you select. Navigtaion is also improved. Eddie shows us how the view is more dynamic, zooming in and out intelligently based on where your next turn is and other information. Maps will give you suggestions for things along the route, and tell you how long it’ll take to reach your destination if you stop at these suggestions. There’s also a Maps extensions API, Eddie’s example is booking a table at a restaurant with an OpenTable restaurant, hailing an illegal taxi cab with Uber, and paying for it all inside Maps. Cool.

Apple Music is also getting updated with an “all-new redesign.” I don’t see the connect tab anymore. Bozoma Saint John went on stage to give a demo. The Music app looks to be streamlined without that connect tab, and pretty different. Lyrics are displayed right below the controls for the music if you scroll down. Very nice. Bozoma tried to get the audience to rap along with Rapper’s Delight, the audience cam showed us various Apple folks attempting to do so, it was extremely embarrassing. The new Apple Music interface looks great, though.

Eddie Cue comes back to discuss Apple News improvements. You can already read this site on there so they’ve done a bit of extra cleanup to improve the rest of the app. For example, they’ve added subscriptions and breaking news notifications.

Craig is back to discuss Homekit improvements. New categories of supported IoT devices have been added. A dedicated Home app is going to be available to control all of your homes. I’m sure monocles are popping out everywhere. Preset scenes will be available to adjust several devices at once. Siri can control these scenes and devices. If you swipe over on Control Center you’ll be able to control devices from there. Notifications will be able to interact with HomeKit devices and display live video from a doorbell device, for example. Apple TV will act as the hub for your HomeKit devices so that you can access them remotely when you’re away from home.

The Phone app will be updated to transcribe voicemails, though this functionality is in beta. An extension API will let other applications interact with the Phone app, in an example slide an extension from Tencent identifies an incoming caller that isn’t in your contacts as a possible spam caller.

When VoIP calls ring through, they’ll integrate into the lockscreen and the phone app just like regular calls. Contacts will be updated to display the options you have to call someone through multiple services. They’ve worked with Cisco to let you get your work calls through your iPhone.

The Messages app has been updated to provide previews of links inline. Videos and photos will display right in Messages. The camera will display a minified camera app inside of messages when you tap that button. Emoji are now three times larger. If you tap the emoji button after typing up a message, but before sending it, it will highlight words that can be replaced with emoji. Message bubbles can now have effects like shrinking or growing to emphasize emotion behind words. Messages can be hidden for a recipient to reveal with a swipe over the text or an image. They’ve also added a short series of icons to send a thumbs up, or a laugh to someone. Apparently we don’t have enough time to select the appropriate emoji now. This demo slide broke and displayed the thumbs-up above the message that it was replying to. You can also send “handwritten messages,” it isn’t clear if these are keyboard input being turned into a generated font with some ink physics or brushed on with a touch, but you can also send “digital touch” messages like drawing a smiley face on the Apple Watch. You can also draw on video or photos with digital touch. There are also fullscreen effects that appear behind the messaging transcript. A slide shows us some fireworks behind the chat bubbles after you send “Happy New Year!”

Bethany and Emron are introduced for a demo. They’re engineering and human interface design team leads. Bethany demonstrated sending and receiving some diffferent message types, Emron received a link to some music that you could play right inside Messages.

iMessage will now support developer-created apps that are launched from an “app drawer” for things like stickers. Craig tells us that some types of apps won’t even require code. Artists can make them a sticker app without coding, but you could also integrate other iOS functionality like the camera. Square Cash will work inside of Messages as well.

Craig started a demo. He demonstrates sending a sticker, and then applying some animated stickers to a photo message. The next demonstration is a group ordering food together through DoorDash, right inside of Messages. Each person selects the food they want from the restaurant (or food truck in this demonstration).

The Mac and Apple Watch will be able to receive these conversations, so it sounds like they won’t be able to work with messaging apps to create messages using them, which makes sense.

Here’s a video they showed to feature the new features in Messages:

Craig finishes up by blitzing through some other improvements to iOS 10. Live editing collaboration in the Notes app. Conversation view in Mail displays messages in a better threaded format. Live photos can be edited and they have stabilization now. Safari on iPad finally lets you use split view to display two websites at the same time. Previously you had to use third-party apps to do that. He also reminds us that many of these new features work for third-party developers.

Finally he re-emphasizes how all of this information that involves computer learning, the memories functionality in Photos for example, will be run on the device. The information that is sent to Apple’s servers will also be kept private by them not building profiles of users. Obviously this is a dig at Google who mines all of your data with no regard for your privacy if it’ll enable their advertisers to have better targeting.

Tim Cook comes back on stage to close things out and emphasize Apple’s commitment to developers. He discusses how great Swift is. He’s talking about how it’s the #1 language project on Github since the source was released. Cook also reiterates how important Swift is as a first language by releasing a new app called Swift Playgrounds for iPad.

Tim invites Cheryl Thomas on stage to demonstrate Swift Playgrounds. The app’s front door screen has lessons and challenges. QuickType suggestions are offered inside of lessons with code suggestions. There’s a simple turtle-style demo of moving a character on the screen with code like moveForward() and collectGem(). Cheryl demonstrates wrapping a for loop around some pre-existing code. This looks like it could give Codeacademy a run for its money, though obviously it’ll be limited to Swift many of the lessons could apply to other languages. We need XCode for iPad.

Cheryl opens a more advanced playground with a new coding keyboard to add new code to the playground.

Tim returns again to announce that Swift Playgrounds will be free, of course. Another video:

That could have been incredibly hokey, but I think they did a great job.

Tim Cook goes back over everything we’ve seen, and closes out the keynote.