On the road to spatial computing
A new paradigm for interacting with the world
Imagine seamlessly blending our virtual and real-world experiences. This is the promise of spatial computing, which offers more natural and intuitive interactions with our digital devices and applications. By better understanding the user’s environment and the user’s state, spatial computing applications move beyond traditional inputs like keyboards and screens, creating interactive spaces that blend the physical and digital world. At its core, spatial computing enables users to manipulate and interact with digital content as they would physical objects, by using gestures, movements and spatial awareness.
Spatial computing is a new paradigm for digital interactions that will revolutionize everything from manufacturing and healthcare to education and architecture. It offers more intuitive ways to harness digital intelligence, whether visualizing data, collaborating remotely with team members, or producing and managing goods and services. It will transform how we interact with the physical world in both our personal and professional lives.
Recognizing its potential to evolve human-computer interactions, major technology companies are investing in spatial computing. With its demands for very low latency edge processing, telecommunication providers see it as a major driver of next-generation architectures. As it matures, spatial computing will become increasingly relevant for any organization that seeks to create more efficient and engaging ways for its customers and employees to operate in our augmented digital world.
What is spatial computing?
Today, we interact with computers through desktop or handheld graphical user interfaces (GUIs), which creates a gap between our digital and physical experiences. We are often forced to choose between engaging with the physical world or the computer screen. Spatial computing (SC) bridges the digital and physical worlds, allowing users to interact with digital content while remaining fully engaged in their physical environment. It brings physical places and objects into computing applications and overlays digital information and functions on them (e.g., hyperlinks, 3D models, sounds, sensor data, etc.).
Ways to interact with the digital-physical world
This new paradigm represents a shift in human-computer interactions towards more bi-directionality: the device not only augments our perception of the world with digital information, the real world provides contextual information to the application, such as the user's current physical context and gestures, which makes interacting with the computing application more natural.
Physical context information can be gathered by cameras, sensors and wearable personal assistants. Consider a cooking scenario where the application provides real-time guidance based on environment analysis—from monitoring pot temperatures using infrared sensors to assessing ingredient freshness. The technology automatically recognizes situations, understands user states (including posture, gestures and mental condition), and processes environmental data (3D geometry, surrounding objects) to deliver timely, relevant assistance. It has the potential to provide real-time input into large AI models that can access the cumulative knowledge of humanity.
Spatial computing application examples
Spatial computing is already being used in industrial and office environments to enhance efficiency and safety. For example, the Nokia Skypark office in Budapest uses spatial computing for precise indoor and outdoor visual localization with dynamic points of interest discovery. The AR-based seat reservation system shows floating status indicators above desks and intuitive indoor navigation to meeting rooms. Other applications provide workplace solutions such as maintenance ticket visualization, equipment-specific video guides, and meeting room availability displays in AR.
Nokia Bell Labs Stuttgart recently presented spatial computing for a radio network planning application. This solution combines 3D mapping with automatic metallic object segmentation and ray tracing to calculate and visualize radio signal heatmaps throughout buildings.
Taqtile's AR technology, which is integrated into the Nokia MX Industrial Edge solution and leverages digital twin information, showcases how spatial computing can provide frontline workers with real-time instructions, IoT data integration, and immediate access to remote expertise. These implementations create a seamless bridge between physical environments and their digital twins, enabling more efficient operations across various scales, from individual objects to entire facilities.
What are the technology enablers for spatial computing?
Spatial computing requires the combination of multiple sophisticated technologies, including advanced sensors, real-time 3D perception, object recognition, and precise localization and mapping systems as well as a number of cloud services. The spatial computing technology stack (pictured) represents a comprehensive framework that bridges the physical and digital worlds.
At its foundation, the physical world (represented in green), encompasses humans, machines, robots, objects, and places, supported by existing enablers (light purple) such as networks, storage, compute resources, and AI/ML capabilities. The framework introduces five crucial new enabling technologies (dark purple): spatial user interfaces, mapping and localization systems, reality modeling and simulation tools, spatial data and contents, and content enablement services. These components work together to create a robust foundation for spatial computing applications.
This technology stack can be applied in consumer, enterprise and industrial applications using the enabling technologies differently to create unique solutions and experiences. How these new enablers function and interact impacts future network requirements and capabilities, as they introduce new demands and opportunities for network infrastructure and services.
User interfaces
The evolution of wearable technology is advancing rapidly, particularly in smart glasses and smart earbuds. From sophisticated mixed reality headsets to lightweight all-day wearables, these devices now feature advanced capabilities including motion sensors, head tracking, vital signal monitoring, and natural language interactions. Modern wearables have become sophisticated sensing platforms that can track location, understand 3D geometry, detect surfaces, and monitor user behaviors including motion, posture, eye gaze and hand movements.
The contextual information generated by wearables can be used by AI-enabled applications to respond dynamically to environmental and user changes. While always-on perception presents new opportunities for AI assistance and automation, it also presents technical challenges. The industry is moving toward a distributed processing approach. For instance, smartphones on our body and powerful computers in the edge cloud can provide the resources needed to analyze and interpret contextual data beyond the capabilities of wearable devices. This distributed perception and intelligence will require very low-latency network services.
Mapping & localization
Modeling & simulation
Data & content
Content services
The evolution of wearable technology is advancing rapidly, particularly in smart glasses and smart earbuds. From sophisticated mixed reality headsets to lightweight all-day wearables, these devices now feature advanced capabilities including motion sensors, head tracking, vital signal monitoring, and natural language interactions. Modern wearables have become sophisticated sensing platforms that can track location, understand 3D geometry, detect surfaces, and monitor user behaviors including motion, posture, eye gaze and hand movements.
The contextual information generated by wearables can be used by AI-enabled applications to respond dynamically to environmental and user changes. While always-on perception presents new opportunities for AI assistance and automation, it also presents technical challenges. The industry is moving toward a distributed processing approach. For instance, smartphones on our body and powerful computers in the edge cloud can provide the resources needed to analyze and interpret contextual data beyond the capabilities of wearable devices. This distributed perception and intelligence will require very low-latency network services.
The process of persistently attaching digital assets to physical locations requires sophisticated vision-based localization technology. This technology enables precise placement and sharing of virtual objects in the real world through a three-step process.
- First, the environment is captured through photos and laser scans, creating a detailed 3D model with unique visual features stored in the "AR cloud".
- Second, users can associate virtual assets with specific physical locations within this model.
- Third, multiple users at the same location can experience these virtual assets in exactly the same position, thanks to highly accurate localization systems.
Reality modeling and simulation cloud services for spatial computing (including but not limited to 3D reconstruction, semantic segmentation and scene understanding) create accurate digital representations of our physical world and simulate how various elements interact within it. The process involves capturing real-world environments through various sensors, processing this data to create detailed 3D models, and implementing physics-based simulations to predict and analyze how different elements—whether they're sound waves, radio signals or physical objects—behave within these digital spaces.
While spatial computing plays a role in capturing digital twins and supplying them with real-time measurements, it also helps in visualizing simulation results in the real world.
Dynamic Digital Twin - Warehouse PoC
Neural radiance field (NeRF) of hockey arena
These technologies are being applied across various domains, from enhanced sports viewing experiences to warehouse inventory management, utilizing cloud processing for complex tasks. As the technology evolves, spatial computing is emerging as a potential primary interface for interacting with digital twins and 3D memories.
Spatial data can encompass everything from environmental data and 3D building models using rich multimedia elements, to real-time information streams and IoT device measurements. Spatially aware, generative AI can dynamically create and adapt content based on specific locations and contexts. The content can be organized into channels and can be filtered, allowing users to customize their experience based on their preferences, subscriptions and current situational needs.
The spatial content extend beyond purely virtual 3D models and 3D sounds transforming traditional media into immersive 3D experiences. This is particularly evident in applications like memory preservation, where smartphones can capture 3D spaces so that mixed reality headsets can replay them, complete with photorealistic avatars and spatial audio.
Viewer outside the room is watching on a mixed reality headset the live stream of a photorealistic avatar captured inside the room.
Large-scale spatial computing applications need a content enablement infrastructure and services that facilitate the creation, discovery, management, distribution and consumption of digital content. Further essential infrastructure components include marketplaces, identity verification systems (potentially leveraging Web3 technology), message brokers for real-time communication, time services that handle synchronization and historical archiving, and potentially many other services that have yet to be invented.
Spatial computing and communication networks
The rapid evolution of spatial computing will necessitate some changes in network architecture, demanding innovative solutions for continuous connectivity, traffic prioritization, and service delivery across multiple technological domains. Spatial computing applications demand continuous bi-directional connectivity to support distributed machine perception, which introduce new challenges for on-body, around-body (e.g., via 5G sidelink and Wi-Fi), and access network technologies. The traffic profile is complex and demanding. Upstream flows include delay-sensitive motion and gesture data, heavy sensor streams for environmental understanding, 3D-map updates, and visual localization requests. Downstream flows include stereo rendered images, volumetric multimedia, dynamic AI-model deployment, and 3D-map chunk distribution.
To maintain optimal performance, spatial computing applications require dedicated network slices for traffic with low delay and jitter requirements. To address these challenges, innovative solutions focused on network application awareness and congestion management are needed. Nokia’s L4S (Low Latency, Low Loss, Scalable throughput) technology provides faster data rate adaptation compared to traditional solutions, thus avoiding latency issues and ensuring reliable immersive XR experiences. The Nokia Network as Code platform offers application developers quality of service on demand (QOD), as well as low-level information exchange with the network via standardized APIs, such as device location and status.
The cellular network infrastructure is expected to expand beyond traditional communication roles to include sensing and localization capabilities. This transformation presents new opportunities for network infrastructure providers and private edge deployments to host essential spatial computing services, including localization, mapping, world analysis and AI inference. Communication service providers (CSPs) and enterprises can evolve into spatial computing service providers, potentially transforming monetization models and expanding developer ecosystems. Critical to this evolution is the development of new standards and interfaces and uninterrupted connectivity across various wireless technologies as well as network and service realms.
Challenges
The spatial computing landscape faces technical and operational challenges that require substantial investment despite uncertain consumer demand.
Devices
Current devices struggle with fundamental limitations in display technology, battery life and form factor constraints. Computing capabilities require specialized hardware solutions for complex processing tasks. Challenges range from heat dissipation issues to the need for lightweight designs that enable comfortable, all-day wearability.
Fragmentation
The spatial computing ecosystem's fragmentation poses another layer of complexity, with various devices, operating systems and positioning methods competing for dominance. This diversity spans most device classes and, even, within them. For instance, glasses, cars and robots each utilize different platforms and positioning techniques from radio to visual positioning. The lack of interoperability between these systems and the absence of standards has become a significant barrier to widespread adoption.
Privacy and security
Privacy and security concerns represent critical challenges in the spatial computing landscape. The integration of always-on cameras, precise location tracking, and 3D mapping of private spaces raises important questions about data ownership, access rights and user privacy. These concerns extend beyond traditional web browsing issues, as spatial computing brings digital tracking and monitoring directly into our physical world, requiring careful consideration of how personal data is captured, stored and shared.
Is spatial the next web?
The evolution of the World Wide Web is moving towards what experts call the Spatial Web, representing a transformative shift from traditional Web paradigms to an immersive, three-dimensional digital experience. The next Web is emerging as a fusion of physical spaces and digital assets, creating an environment where the internet surrounds us in our daily lives. This evolution is already visible in current technologies, such as Google’s and Apples’s AR cloud implementations, which allow users to place and interact with digital content in real-world locations.
As we move toward what some call the Spatial Internet—where our physical world is enriched with location-specific information—new challenges emerge in content enablement and service discovery. This evolution requires continuous geospatial service discovery mechanisms that can identify available services and relevant content around users, personalized search capabilities, and content filtering systems that ideally aren't controlled by a single entity.
Drawing inspiration from the World Wide Web's successful design principles, the future of the Spatial Web must embrace openness and interoperability, avoiding centralized control in favor of interconnected platforms. As Tim Berners-Lee noted, information systems naturally evolve from small, isolated beginnings to merged, interconnected networks. This democratic approach to content creation and consumption, where anyone can contribute and access spatial experiences without central oversight, will be crucial for mass adoption and the sustainable growth of the Spatial Web ecosystem, much as the traditional Web has flourished through its decentralized nature.
Spatial computing intertwined with the network will emerge as the connective technology between the physical, digital and human worlds. With an open and interoperable structure, data privacy and security, and the evolution of our devices to understand the physical world around us, spatial computing has the potential to be the arena in which AI assistants truly become integrated into our daily lives, both at work and at home. Just as the internet revolutionized almost every aspect of life and culture, so will the Spatial Internet. As with all nascent technologies those entities that participate in the early development of spatial computing will have the greatest power to shape its ultimate evolution.
