taylor.town
about now spam rss

AR Demands Peripherals

AR peripherals

AR devices are windows into the wet world.

Seeing the wet world is easy. But as of 2023, manipulating wet stuff is clunky.

I suggest (1) using gaze detection for pointing and (2) AR peripherals for manipulating.

Touch Screen World

The most intuitive AR interface would be a “touch screen world”. The “touch screen world” already exists in a limited capacity on Oculus devices.

If you want to touch something in AR/VR, just touch it, and onboard cameras will interpret your gestures as manipulation.

You could “touch” far away objects by pointing your finger at them, and “click” by moving your thumb. Pinching gestures are a more precise alternative.

Highlights/tags/annotations will become wet web affordances (think of glowing items in RPGs).

Developers will make tailored interfaces for many objects. Bananas will become smartphones.

AR Peripherals

The “touch screen world” model is an intuitive but insufficient interface for the wet web:

People need peripherals for (1) tactile feedback, (2) typing, and (3) precise input.

I imagine 3 classes AR peripheral use:

Gaze Detection

Watch yourself use a mouse or trackpad. Notice that you never click on things without looking at the cursor.

The whole point of the cursor is to click at what you’re looking at. It’s for cursory action.

With good gaze detection, a cursor is redundant. Your eyes are mice.

Your eyes cross when looking at nearby objects and parallelize when looking far away. This can be used for crude 3D pointing that would be difficult with a peripheral.

Gaze detection can be used for pointing, and AR peripherals and/or gestures can manipulate what you’re looking at.

If your eyes are mice, then you will never look at your peripherals while using them. Peripherals must be simple enough to stay outside your periphery.

Most people are less productive on touchscreens, but use them for convenience and portability. Gaze detection fits a similar niche for AR interactions. It’s not the preferred mode of interaction, but it’s probably the most convenient. Nobody wants to carry a mouse.


Related AR/VR essays: Apple Will Win The AR/VR Wars, Bananas Will Become Smartphones, Monomode and Multimode in Augmented Reality, Claim a Domain in the Wet Web, Tools and Techniques for AR/VR Media, AR Interoperability Opportunities