I would really like to be able to interact with applications in a simultaneous multi-modal fashion, taking advantage of pointing device, keyboard, voice, and gestures. In some cases, I'd like to direct speech commands to a background app while pointer and keyboard focus remains in the front app. Sometimes you need information from an app, not necessarily to use it.
Say I’m chatting with a friend to schedule dinner next week, but I don’t remember my schedule. Calendar is open and visible in another region of the screen, but it’s showing the current week. Assuming the Mac is always listening for the Siri invocation command, I just say “Hey, Siri, go to next week in Calendar” rather than switching to Calendar, navigating to next week, then switching back to the chat. This scenario is depicted in the video below.
Siri Background App Control
Speedy interpretation of speech commands is crucial to these interactions feeling fluid and natural. Incorporation of a speech co-processor (as with the iMac Pro) allows Macs to always listen for the “Hey, Siri” prompt rather than having to invoke Siri through the menubar item or keyboard shortcut.
If Apple builds a display (still hoping for a 40-inch 8K) to pair with the forthcoming Mac Pro, they should include the dot field hardware that enables Face Unlock on the iPhone X. Provide that on the Mac, but go further by using it to recognize hand gestures made in the space between the user and the display. It could also potentially be useful for those with motor impairments as a way to use blink patterns to execute commands. Maybe it could even be used as a way to translate sign languages to text, without having to wear a special glove.
Hold up a thumbs up or down, or one or more fingers to star rate the playing song
Raise or lower your hand to control audio or video volume
Make a pinch in or out gesture in mid-air to zoom in mapping or graphics apps; not everyone has a trackpad
Make a “holding a camera, pressing the shutter button” gesture to take a screenshot
Once you’ve lived in New York for a while, you don’t necessarily need a specific route to get between two points. You know roughly which subway lines will you get from point A to point B. The important thing becomes knowing which trains are operating normally and which are delayed or rerouted (which seems pretty common these days). To that end, the MTA site provides a status widget providing an overview, but I’d like it to be a single click (or two-finger swipe) away in the Today view of the Notifications Center. A similar display would also be useful on phones.
Here's the UI as seen in Safari Technical Preview 41, released October 4, 2017.
Revised Responsive Design Mode
Add a bandwidth throttling menu:
Improve the contrast of the selected viewport setting in a style consistent with the Web Inspector
Show a rotate icon when hovering on a device icon
Align label baselines
Add a screen mode menu for iPads that would allow selection from:
This provides an explicit UI for using the iPad multi-tasking modes and would allow users to switch between portrait and landscape orientations of different iPad models while staying in a multi-tasking view. In STP 41, you can skip cycling through the iPad multi-tasking modes by Option-clicking the iPad icon.
I like the announced specs for the iMac Pro, but I’m waiting for Apple to unveil a large external display. I have a 27″ 5K display and I want more space for photos and UI designs without having to deal with multiple displays. Check out Dell’s 32″ 8K display. According to Sven Neuhaus’ PPI calculator, the Dell’s PPI is about 280, which is not ideal for macOS. To fit 8K of pixels at the 220 PPI that makes for comfortable rendering of the macOS interface, Apple would need to build a 40″ diagonal panel. Sounds good to me.
The other thing I’m really looking forward to is APFS in High Sierra. I had to reformat an external drive recently because of corruption that likely could have been avoided with a modern filesystem.
Travis Kalanick, the chief executive of Uber, visited Apple’s headquarters in early 2015 to meet with Timothy D. Cook, who runs the iPhone maker. It was a session that Mr. Kalanick was dreading.
For months, Mr. Kalanick had pulled a fast one on Apple by directing his employees to help camouflage the ride-hailing app from Apple’s engineers. The reason? So Apple would not find out that Uber had secretly been tracking iPhones even after its app had been deleted from the devices, violating Apple’s privacy guidelines.
But Apple was on to the deception, and when Mr. Kalanick arrived at the midafternoon meeting sporting his favorite pair of bright red sneakers and hot-pink socks, Mr. Cook was prepared. “So, I’ve heard you’ve been breaking some of our rules,” Mr. Cook said in his calm, Southern tone. Stop the trickery, Mr. Cook then demanded, or Uber’s app would be kicked out of Apple’s App Store.
For Mr. Kalanick, the moment was fraught with tension. If Uber’s app was yanked from the App Store, it would lose access to millions of iPhone customers — essentially destroying the ride-hailing company’s business. So Mr. Kalanick acceded.
I expect nothing less than sociopathy from Kalanick and Uber. This also confirms that there are different rules for different apps on Apple’s App Store. If you’re big enough, you can clearly violate the rules and get away with a reprimand from Tim Cook.
I’m glad Apple announced that they will be releasing both a new professional desktop and an external display. Prompted by Michael Tsai, Justin Williams, and Marco Arment and my own experience with Apple’s current (meaning wildly outdated) pro desktop, a few thoughts on future pro hardware. While my current cylindrical Mac Pro works reasonably well, redrawing to the 5K display I’m using (a now-discontinued Dell UP2715K) is janky when switching spaces or using Exposé. The display output is also sometimes horribly distorted after waking from sleep, requiring the monitor’s power cord to be detached and reattached. Not exactly “It just works”. At least it doesn’t turn off near a WiFi router. This is the third panel I received from Dell. The first one they shipped me had a horizontal line of dead pixels, then decided to render the two sides of the screen with different color saturation. The second display they shipped me had both a few dead pixels and a small scratch. To Dell’s credit, their customer service was pretty good. That said, I have never had this kind of problem with brand new display quality from Apple.
My wishes for the new Mac Pro are basically the same as Justin’s: Make it fast, include more memory in the baseline configuration, and let us max out a custom build with the most powerful GPU available. nVidia has released beta Mac drivers for their Pascal-based models, which means that at least some of the work to support them on the Mac is already done. Thunderbolt 3 (or 4) and USB-C are nice, maybe all with DisplayPort 1.4 support for driving 8K/60Hz panels.
8K Cinema Display
I hope Apple’s new stand-alone display is at least 8K resolution for all those who want to work on photos and video at as close to full size as possible while still having screen real estate for UI. 8K resolution would also provide differentiation with the 5K iMac. If Dell can produce an 8K panel with 98% of the DCI-P3 color space, surely Apple can offer something competitive. Ideally, it would be taller. If you used a 30″ Cinema Display (2560×1600), then moved to one with the more common 2560×1440 points (the UP2715K is double that in pixels), you miss the additional 160 points of height.
When resizing objects in Sketch, I’d like to see the increment (or decrement) value along with the dimensions.
The change value is always relative to the dimensions before the drag began, so if you increased the width of the example above by 10, then decreased that by 5 without releasing the drag handle, the counter would read +5.
If they prove distracting, maybe a “Show dimension change counts” checkbox in the Layers section of Preferences.
Ideally, these would display when resizing using the keyboard as well.
Throw typography nerds a bone by using a proper multiplication symbol rather than an x for the width/height separator. I assume you can add it from the Characters palette to the appropriate place in Xcode.
I’m really happy with how the new (nestable!) symbols are shaping up in the Sketch 3.7 betas — great work by the team at Bohemian Coding.