[U43] Spatial Computing: is this a hammer looking for a nail?
Apple bets its developer ecosystem will create the ultimate user experiences
Before Apple launched the iPhone in 2007, CEO Steve Jobs had already identified and defined a problem. Consumers needed multiple mobile devices to complete routine daily tasks with sub-optimal user interfaces on inferior computing platforms. Steve Jobs aimed to design a mobile device that was 1.) easy for anyone to use and 2.) performed more like a networked computer than a phone. This solution eventually transformed the mobile experiences for voicemail, phone calls, photos, contacts, calendars, email, audio, banking, weather, navigation with real-time maps, internet search, payments and many other applications.
VR will become something everyone wants before it becomes something everyone can afford. December 2015 tweet by Palmer Lucky, Founder & CEO, Oculus VR (acquired by Meta Platforms)
Adoption for the original iPhone in 2007 was limited to 1.4 million units generating US$630 million in sales. But the problem was clearly identified and the solution was introduced for innovators and early adopters. Sales increased to 11.6 million units in 2008 and increased significantly each year with 231.2 million units sold in 2015. Ongoing improvements to the cameras, rechargeable battery and third-party Apps continue to drive iPhone sales. Apple’s growing ecosystem anchored by the iPhone drives user retention with services like Apple Pay, Music and Fitness+ and connected devices like AirPods and the Apple Watch.
iPod predated the iPhone to innovate the MP3 player use case for 1,000s of digital audio files stored on a device.
iTunes software made most songs available for iPod and iPhone users with Apple negotiating contracts with the recording labels so users could access a single song for a one-time payment of $0.99 and whole records for $9.99. This service expanded to include streaming with Apple Music.
iPhone used a unix-based operating system (OS X) to develop and run applications and operate core functions on the device. This became iOS for the iPhone.
iPhone introduced multi-touch finger gestures to eliminate the need for a separate stylus for the user interface: swipe, tap, pinch and reverse-pinch.
3 separate mobile devices including an iPod, internet communicator and mobile phone were consolidated as features to realize the original iPhone device.
Ongoing mobile App innovation unlocked more capabilities for the iPhone hardware to disrupt more products and services with software downloaded from the App Store.
iPad product line was added to expand use for iOS beyond the iPhone.
iPhone Pro, iPhone Pro Max and iPhone SE models were added to provide multiple tiers for performance, size and price beyond the original iPhone form factor. Prior product generations are also offered to lower price even further.
When Apple revealed the 1st generation Vision Pro in June 2023, what problem had CEO Tim Cook identified and defined for a novel computing platform that extends reality? Apple referred to this as “spatial computing”. So far, the target problem appears to be limitations and side-effects associated with using existing virtual reality (VR) technology. These include motion sickness, isolation during use, and the need for auxiliary input devices like hand controllers.
But, why do we need to wear the Vision Pro in the first place for more than a product demo?
Tim Cook expects teams at Apple and Disney will design and develop the ultimate user experiences for consumers who subscribe to Disney+.
Apple is also betting its extensive developer ecosystem will innovate 2D and 3D experiences to solve multiple problems while the Vision platform evolves with multiple products and services at different price points and levels of performance.
What is Spatial Computing?
Spatial computing was proposed by Simon Greenwold in his 2003 Master of Science in Media Arts and Sciences thesis at MIT. He defined the term as human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces. Spatial computing today spans industrial robotics to automate tasks, autonomous navigation for self-driving cars, virtual reality headsets and haptic feedback using wearable devices.
Spatial computing introduces the first Apple product you look through. However, this is really pass-through video with multiple cameras acquiring images, low-latency computing processing these images, and a high-resolution display for each eye to realize binocular vision. Apple’s ambition is a spatial computing platform to sustain innovation beyond the Macintosh, iPhone and iPad.
What spatial computing capabilities will drive adoption for 100s of millions of consumers with appropriate pricing? This requires attention to specific problems spatial computing is best positioned to solve. Palmer Lucky who founded Oculus VR (platform acquired by Meta Platforms) has stated that he agrees with the product positioning Apple chose. To paraphrase Palmer, before Vision Pro is a product more people can afford, it has to become a product people want.
An immediate proxy for advanced spatial computing might be self-driving cars. A human operator can rely on GPS and Google Maps running on their iPhone to navigate from point A to point B. But this is not adequate for self-driving cars to avoid hazards human operators can manage in real-time with adequate attention. Real-time computer vision, awareness of surroundings, and route planning are critical for true self-driving. Autonomous vehicles are continuously using an array of sensors for real-time 3D mapping to calculate the optimal path.
Closed-loop spatial computing for self-driving will eventually enable transportation without a human operator. Apple Vision Pro is open-loop spatial computing we wear and control with hand gestures, eye tracking and voice. Both products have many unique capabilities, but require ongoing innovation to advance use beyond innovators and early adopters.
Metaverse
Meta Platforms (formerly Facebook, Inc.) CEO Mark Zuckerberg acquired Oculus VR in 2014 to add a leading hardware and immersive VR platform to Meta’s assets. Meta rebranded from Facebook in 2021 to pursue the metaverse where users spend time in a virtual world while wearing a VR headset. Meta has developed applications to support workplace collaboration, but gaming is the primary use case for the metaverse today.
Meta acquired Beat Games, the company behind the popular VR game Beat Saber in 2022. Digital assets, content and experiences are key features to drive adoption of the metaverse. McDonald’s envisions teenage customers meeting in the metaverse, ordering food and transacting the purchase for delivery from a local restaurant. Digital payments are another key feature of the metaverse.
The Meta Quest 3 will launch in the fall of 2023. Meta recently dropped the price of the Meta Quest 2 to $299. Meta will benefit from Apple driving performance with a higher priced platform. Meta can learn from what is working well on Vision Pro to build out less expensive alternatives.
Many consumers who eventually try the Vision Pro will purchase a more affordable Meta product instead. Android phones took market share with more affordable pricing from early generations of the iPhone. But Apple used the iPhone to build an ecosystem with multiple revenue streams beyond hardware sales. Apple likely sees the same long-term opportunity with spatial computing.
What problem can Spatial Computing solve for consumers wearing a $3,500 Vision Pro?
Many professional and collegiate sports teams have fans located worldwide. These range from casual fans to fanatics with disposable income who will pay top dollar for playoff tickets to sit in the best seats. The inconvenience of living thousands of miles from the venue may present the most significant barrier to attending these events versus the actual cost of tickets to attend the event in-person.
Entertainment: Premium Access to Select Events
Disney has another asset known as ESPN. Instead of a fan paying StubHub $10,000 for the best seat to attend one playoff game, ESPN could offer a $6,000 playoff package for all the playoff games from the same spatial perspective as the best seat in the arena or stadium. If your favorite team is eliminated along the way, you could either continue to watch the rest of the playoffs or receive a refund for the prorated portion of playoff rounds eliminated for your team.
If the Boston Bruins are eliminated in the first round of the Stanley Cup playoffs, you can request a pro-rated refund ($4,500: 3/4 of $6,000) for not watching the next 3 playoff rounds. However, another fan can pay the full amount if they watch the Stanley Cup playoffs all the way through the Finals for $375 per game if they watch the minimum 16 out of 16 games. ESPN would need to contract with the respective league (NHL in this case) on a revenue share arrangement for this service, but it would increase the “premium” attendance of events.
If you take all the top college and professional sports teams playing Football, Basketball, Baseball, Hockey and Soccer plus Golf, MMA, Boxing, NASCAR, Formula One and Grand Prix racing, I’m sure fan fanatics would pay a premium for a superior Vision Pro experience “package” when they are already paying a similar premium to attend one game or event in-person. Concerts, movie releases, comedy, opera, symphonies and ballet offer similar opportunities.
The experience would need to exceed anything else available today to command premium pricing. This will require camera and sound equipment to acquire video in 3D to also view it in 3D. As the technology improves, access to more events will expand. Remote attendance eliminates the time and cost to travel to a whole series of events. This expands the market for premium access to select events.
Entertainment: Global Destinations
Travel overseas provides the opportunity to experience many new places and cultures. But the distance and time required has led many airlines to offer a premium experience to reduce the burden for flights that may require 16 or more hours to complete. Lay-flat seats to sleep and premium meals are features of first-class seating. These options can also cost over $10,000 per seat for round-trip, first-class travel. Factoring in the time commitment, international travel presents a number of burdens when you also factor in the cost of hotels and meals while away from home. These do complement the experience, but likely limit the number of trips most people can take annually.
Popular tourist destinations like the Colosseum in Rome, Eiffel Tower in Paris, Taj Mahal in India, and Great Barrier Reef off the coast of Australia could provide initial opportunities with existing digital assets. But long-term, 3D digital assets to help reproduce a guided “walking tour” in various destinations would offer more significant immersive experiences.
Entertainment: Unique Experiences
Spatial computing can also offer unique experiences for popular 3D movies like Avatar and Gravity. Trailers for specific scenes could demonstrate the possibilities with spatial computing while full 3D movies are optimized for the Vision Pro platform. Spatial computing also provides a unique environment to watch and interact with existing 2D movies. This likely bridges the gap between 3D content becoming more of the standard.
Climbing a mountain like K2 or diving to an ocean depth to see the Titanic require significant fitness and risk, but also provide significant views. Climbing K2 could be accelerated into a 90-minute immersive experience vs. over 40 days plus international travel with a total climbing expedition cost of over US$60,000. Swimming with humpback whales off the coast of Tonga also blends travel with a unique visual and audio experience. This also provides a teaching opportunity to make more people aware of the challenges whales face.
What Apple is not prioritizing for now?
Simulation and Training are applications for virtual reality Apple has not highlighted so far for spatial computing. Flight simulation may require graphic capabilities beyond what Vision Pro can provide at this time. And these applications may also require more specialized development of content versus repurposing existing content for entertainment.
Gaming is a key use-case for Meta and other virtual reality platforms. Apple is likely avoiding this area to help direct developers to areas where Vision Pro can move beyond existing virtual reality platforms. Ready Player One is acknowledged in the Vision Pro video below when the individual puts on the hardware and we view what they see. But what this individual sees is a passthrough view of their room versus the Oasis.
Health and Fitness have not been highlighted so far, but will likely be a big opportunity for Vision Pro in the future. Meditation and mindfulness are well-suited for immersive environments. Vision Pro can also expand into medical uses such as treating pain, neuro-visual disorders, and mental health with the proper regulatory clearance. The current price will be a barrier, but this will be addressed as the technology evolves and scales unit production to lower input costs.
Meet the Apple Vision Pro
I think some design features are important regarding Apple’s new spatial computing platform. I summarize those below. Vision Pro isn’t a “me-too” product design with incremental improvements beyond existing products. Apple reimagined the hardware and software requirements and technical specifications for a 1st generation product to explore and define spatial computing.
I anticipate as Vision Pro is launched, an enthusiast community will emerge and evolve to demo and comment on the latest released version of Apple’s spatial computing environment similar to the communities that evaluate and comment on released versions of other advanced technology like self-driving software for electric vehicles. Content Creators could be key innovators and early adopters to accelerate the adoption of spatial computing.
This Content Creator group will provide immediate feedback on what is working and what needs to improve for the best spatial computing experience. New applications can also be evaluated, so recording these experiences to share on social media platforms will be critical for Apple to leverage this opportunity.
Some key insights into the Apple Vision Pro.
Apple did not mention terms like virtual reality or the metaverse when it revealed the Vision Pro. Apple wants to control the narrative with its own terms like spatial computing “environments”. Apple also didn’t make comparisons to products from other companies. It is creating a new category for Apple and will reference the evolution of its own platform. However, Apple did chose to position this 1st generation product to enable immersive experiences to compete with virtual reality alternatives.
Apple’s rumored production volume for the $3,499 Vision Pro was originally 1 million units, but with key supply chain challenges for some key components this will likely be more like 300-400 thousand units. This provides adequate production to incentivize the supply chain for components and to seed the developer market with devices, 1st generation Apps and available content to explore the platforms capabilities. The 1st generation hardware price is high to drive performance. This is key to attract the best developers to create the best possible user experiences.
Controlling the Vision Pro is possible using only your hands, eyes and voice. Selecting an icon in the field of view is as a simple as looking at it and confirming the selection with a finger gesture. But as Apple advances spatial computing, totally new ways to navigate environments and content will emerge. This may present one of the biggest opportunities for innovation beyond current input hardware like keyboards and trackpads.
An extensive sensor array and real-time spatial computing with visionOS capture the surrounding environment, hand movements, facial features, eye tracking and voice using dual-processors. These include the M2 processor (system-on-a-chip used in the MacBook Air and iPAD Pro) and the new R1 processor (extended reality by processing the sensor array). The performance using dual-processors minimizes latency to reduce motion sickness.
Vision Pro cameras allow the user to capture images in 3D for viewing later using spatial computing. The cameras also capture when someone enters your immediate surroundings while you are using spatial computing so you are aware someone else is present in case they need your attention. The Vision Pro will also communicate to others whether you are in an immersive environment by not showing an image of your eyes and then showing your eyes when you are in passthrough mode to see your environment.
Power for the Vision Pro uses an external rechargeable battery pack. These packs will likely be swappable in the future since the current pack can only power the Vision Pro for about 2 hours of continuous use. This design was likely chosen to reduce some constraints from what is worn on the head. This will be important to offer more flexibility in the future.
Final Thoughts
The Apple Vision Pro is an advanced product design to define an emerging computing category to enable more immersive and augmented experiences. But more so than previous product introductions from Apple like the Macintosh and iPhone, the 1st generation Vision Pro is more of a discovery platform for leading innovators and early adopters to experience, build and advance concepts.
Vision Pro is also a clear “hammer” for Apple with 1,000s of patent applications filed so far to protect its intellectual property. And the “nail” is Meta Platform’s ambition to add a significant hardware platform to its social media assets like Facebook, Instagram, WhatsApp and Messenger with over 2 billion users worldwide.
Vision Pro may also lead to a product dead end just like Microsoft’s Hololens 3 for mixed reality. No one has been able to build and scale a hardware platform for viewing content that you wear on your head for sustained use and wide consumer adoption. Success has been limited to niche uses with a clear problem to solve. Apple may conquer the technical hurdles to drive more widespread adoption, but they do not need to conquer this space to win. They just need to drive some of the key technology to keep Meta in check.
Apple is also betting its developer ecosystem inside and outside the company and key partners like Disney will identify problems spatial computing is ideal to solve for general consumer adoption. The Vision Pro will not be available until 2024, so feedback from product demos and early production runs may influence the final design and product roadmap. This will continue as the product launches. Apple customers, developers and partners will influence how the product is used and will prioritize features on the product roadmap for future product generations.
Price will be important to influence what innovation is in scope. Apple is targeting niche addressable markets that total $3-7 billion in aggregate assuming 1-2 million units at $3,499 per device. The Vision Pro could be the beginning of an important category that expands with multiple products at different levels of price and performance. Apple may even try to acquire Disney in the future to combine its studio intellectual property and content creation capabilities with the spatial computing technology only Apple could innovate over time.
Apple design and performance will still command premium pricing even for a product positioned for the general consumer market, but such a product will likely be in the price range of an iPhone Pro Max at $1,200 versus the current Vision Pro price of $3,499. The lower-priced product may also be more aligned with augmented reality glasses on the go versus goggles to enable more immersive experiences. An intermediate Vision Pro product priced around $2,500 may eventually replace the current Vision Pro. A $6,000+ Vision Pro Max product could also be positioned to drive superior performance for the ultimate immersive spatial computing experiences.
Best,
Stephen
Nothing in this Update is intended to serve as financial advice. Do your own research. The opinions and views expressed in this newsletter are those of the author. They do not purport to reflect the opinions, views or policies of any other organization, company or employer.