Hidden Interfaces for Ambient Computing


As shopper electronics and internet-connected home equipment have gotten extra widespread, houses are starting to embrace numerous kinds of linked gadgets that provide performance like music management, voice help, and residential automation. A swish integration of gadgets requires adaptation to present aesthetics and person types moderately than merely including screens, which might simply disrupt a visible house, particularly after they develop into monolithic surfaces or black screens when powered down or not actively used. Thus there’s an rising want to create linked ambient computing gadgets and home equipment that may protect the aesthetics of on a regular basis supplies, whereas offering on-demand entry to interplay and digital shows.

Illustration of how hidden interfaces can seem and disappear in on a regular basis surfaces, corresponding to a mirror or the wooden paneling of a house equipment.

In “Hidden Interfaces for Ambient Computing: Enabling Interplay in On a regular basis Supplies by way of Excessive-Brightness Visuals on Low-Price Matrix Shows”, offered at ACM CHI 2022, we describe an interface know-how that’s designed to be embedded beneath supplies and our imaginative and prescient of how such know-how can co-exist with on a regular basis supplies and aesthetics. This know-how makes it doable to have high-brightness, low-cost shows seem from beneath supplies corresponding to textile, wooden veneer, acrylic or one-way mirrors, for on-demand touch-based interplay.

Hidden interface prototypes exhibit shiny and expressive rendering beneath on a regular basis supplies. From left to proper: thermostat below textile, a scalable clock below wooden veneer, and a caller ID show and a zooming countdown below mirrored surfaces.

Parallel Rendering: Boosting PMOLED Brightness for Ambient Computing
Whereas lots of in the present day’s shopper gadgets make use of active-matrix natural light-emitting diode (AMOLED) shows, their value and manufacturing complexity is prohibitive for ambient computing. But different show applied sciences, corresponding to E-ink and LCD, don’t have adequate brightness to penetrate supplies.

To handle this hole, we discover the potential of passive-matrix OLEDs (PMOLEDs), that are primarily based on a easy design that considerably reduces value and complexity. Nonetheless, PMOLEDs sometimes use scanline rendering, the place energetic show driver circuitry sequentially prompts one row at a time, a course of that limits show brightness and introduces flicker.

As a substitute, we suggest a system that makes use of parallel rendering, the place as many rows as doable are activated concurrently in every operation by grouping rectilinear shapes of horizontal and vertical traces. For instance, a sq. will be proven with simply two operations, in distinction to conventional scanline rendering that wants as many operations as there are rows. With fewer operations, parallel rendering can output considerably extra gentle in every on the spot to spice up brightness and remove flicker. The approach just isn’t strictly restricted to traces and rectangles even when that’s the place we see essentially the most dramatic efficiency enhance. For instance, one might add further rendering steps for antialiasing (i.e., smoothing of) non-rectilinear content material.

Illustration of scanline rendering (prime) and parallel rendering (backside) operations of an unfilled rectangle. Parallel rendering achieves shiny, flicker-free graphics by concurrently activating a number of rows.

Rendering Person Interfaces and Textual content
We present that hidden interfaces can be utilized to create dynamic and expressive interactions. With a set of basic UI parts corresponding to buttons, switches, sliders, and cursors, every interface can present totally different fundamental controls, corresponding to gentle switches, quantity controls and thermostats. We created a scalable font (i.e., a set of numbers and letters) that’s designed for environment friendly rendering in just some operations. Whereas we at present exclude letters “okay, z, x” with their diagonal traces, they may very well be supported with further operations. The per-frame-control of font properties coupled with the excessive body fee of the show allows very fluid animations — this functionality tremendously expands the expressivity of the rectilinear graphics far past what is feasible on fastened 7-segment LED shows.

On this work, we exhibit numerous examples, corresponding to a scalable clock, a caller ID show, a zooming countdown timer, and a music visualizer.

Realizing Hidden Interfaces with Interactive {Hardware}
To implement proof-of-concept hidden interfaces, we use a PMOLED show with 128×96 decision that has all row and column drivers routed to a connector for direct entry. We use a customized printed circuit board (PCB) with fourteen 16-channel digital-to-analog converters (DACs) to immediately interface these 224 traces from a Raspberry Pi 3 A+. The contact interplay is enabled by a ring-shaped PCB surrounding the show with 12 electrodes organized in arc segments.

Comparability to Current Applied sciences
We in contrast the brightness of our parallel rendering to each the scanline on the identical PMOLED and a small and enormous state-of-the-art AMOLED. We examined brightness by way of six widespread supplies, corresponding to wooden and plastic. The fabric thickness ranged from 0.2 mm for the one-way mirror movie to 1.6 mm for basswood. We measured brightness in lux (lx = gentle depth as perceived by the human eye) utilizing a gentle meter close to the show. The environmental gentle was saved dim, barely above the sunshine meter’s minimal sensitivity. For easy rectangular shapes, we noticed 5–40x brightness enhance for the PMOLED compared to the AMOLED. The exception was the thick basswood, which didn’t let a lot gentle by way of for any rendering know-how.

Instance exhibiting efficiency distinction between parallel rendering on the PMOLED (this work) and a equally sized trendy 1.4″ AMOLED.

To validate the findings from our technical characterization with extra lifelike and sophisticated content material, we consider the quantity “2”, a grid of checkboxes, three progress bars, and the textual content “Good Life”. For this extra advanced content material, we noticed a 3.6–9.3x brightness enchancment. These outcomes counsel that our method of parallel rendering on PMOLED allows show by way of a number of supplies, and outperforms widespread state-of-the-art AMOLED shows, which appear to not be usable for the examined situations.

Brightness experiments with further shapes that require totally different numbers of operations (ops). Measurements are proven compared to giant state-of-the-art AMOLED shows.

What’s Subsequent?
On this work, we enabled hidden interfaces that may be embedded in conventional supplies and seem on demand. Our lab analysis suggests unmet alternatives to introduce hidden shows with easy, but expressive, dynamic and interactive UI parts and textual content in conventional supplies, particularly wooden and mirror, to mix into folks’s houses.

Sooner or later, we hope to research extra superior parallel rendering methods, utilizing algorithms that might additionally assist pictures and sophisticated vector graphics. Moreover, we plan to discover environment friendly {hardware} designs. For instance, application-specific built-in circuits (ASICs) might allow an affordable and small show controller with parallel rendering as an alternative of a giant array of DACs. Lastly, longitudinal deployment would allow us to go deeper into understanding person adoption and habits with hidden interfaces.

Hidden interfaces exhibit how management and suggestions surfaces of good gadgets and home equipment might visually disappear when not in use after which seem when within the person’s proximity or contact. We hope this route will encourage the neighborhood to think about different approaches and situations the place know-how can fade into the background for a extra harmonious coexistence with conventional supplies and human environments.

In the beginning, we want to thank Ali Rahimi and Roman Lewkow for the collaboration, together with offering the enabling know-how. We additionally thank Olivier Bau, Aaron Soloway, Mayur Panchal and Sukhraj Hothi for his or her prototyping and fabrication contributions. We thank Michelle Chang and Mark Zarich for visible designs, illustrations and presentation assist. We thank Google ATAP and the Google Interplay Lab for his or her assist of the mission. Lastly, we thank Sarah Sterman and Mathieu Le Goc for useful discussions and ideas.


Please enter your comment!
Please enter your name here