���˴�Ƭ

Imagine you could monitor your household’s water consumption by glancing at a painting of a waterfall and noting how much water was cascading onto the rocks below. No longer just a painting, this interactive home aesthetic might also encourage your household to think more about water usage − perhaps resulting in behavioural changes.

And what if a lamp could do the same, but with regard to energy consumption? A glance at its interactive shade could inform those living in the home about how much the lamp had been used and, consequently, provide a visual indication of the household’s energy consumption.

Aside from prompting us to think about our use of precious resources such as water or energy, what potential could these items have for the psychological state of our homes? What if, for example, a rug could sense the level of noise in the house – particularly the screaming, crying, loud or harsh language – and, over time, visualize this information by changing its shape? A flatter rug might reflect an atmosphere of tranquility, while a crinkled-up, contorted rug might reflect a more turbulent household. Could this provide a visual prompt for the household to modify noise levels?

These are just a few examples drawn from an IKEA-like catalogue filled with household objects that “sense and interact in order to visualize unseen data and behaviour of family members,” says Dr. Sara Nabil, who has a PhD in human computer interactions (HCI) computing and is an assistant professor at Queen’s University’s School of Computing as well as head of the university’s iStudio research lab.

Decoraction: A Catalogue for Interactive Decor of the Nearest-Future is a collaborative project between Dr. Nabil and Dr. David Kirk, professor of digital living at Newcastle University in the U.K. with the advice of Northumbria Design School’s Dr. Julie Trueman. The collaboration took place between 2017 and 2020 and was published in February 2021 at the International Conference of Tangible, Embedded, and Embodied Interaction (TEI’21).

The catalogue explores how interaction can occur between humans and “interactive materials that have sensing and actuating properties,” Dr. Nabil explains − the name itself referring to “decor” and interaction or action of the decor.

“My work with Sara has been hugely engaging and has given us the opportunity to really push creative thinking around interactive futures,” says Dr. Kirk. “The work on Decoraction is truly groundbreaking, opening up discussions around how we might design interactivity into everyday objects and materials. This really opens up the possibilities of how we might interact with technology in the future.”

Decoraction uses “pre-existing technology,” often aesthetic objects in the home, such as wall hangings, lampshades, objets d’art, furniture and rugs, and “intersects them with recent aesthetic IoT devices [internet of things devices, such as smart appliances],” explains Dr. Nabil. This might include using sensors, conductive thread that transmits data/energy, or thermochromic pigments and photochromic paints that change properties with heat and light.

What Decoraction does not include? Gears, motors, or bulky wires. Rather, the technology providing the interactivity is woven seamlessly into the objects.

For example, in Decoraction, Dr. Nabil and Dr. Kirk designed Despot, a lamp with spots painted on the shade’s fabric in blue thermochromic paint, which disappear when heat emits from the lamp’s bulb. As the catalogue notes, “the more spots vanish from the lampshade, the more it raises occupants’ awareness of their energy usage.”

The aforementioned painting that responds to the home’s water use – this is drawn from the Decoraction catalogue as well, referred to as Waterfall, which is, as one might expect, a picture of water cascading onto rocks. As Dr. Nabil explains, the artwork could be connected to a smart meter in the home that measures household water consumption. The readings from the meter are accessed by actuators in the painting, which alter the cascading water displayed through the use of thermochromic paint. The meter’s readings relay information to the painting; this data heats or cools the wall art, and the heat-sensitive paint responds to the change in temperature. In doing so, the painting displays more or less water.

One might be curious as to the style of the painting, since the aesthetic value of the object in Dr. Nabil and Dr. David Kirk’s approach in Decoraction is not overlooked. As the catalogue states, “This multifaceted painting relies on an impressionistic approach that is better suited to interior aesthetic experiences.” Think French impressionist Claude Monet.

“By weaving interactivity into these artifacts, we have to think about aesthetics; otherwise they wouldn’t be acceptable to the potential owners, but also importantly the interactivity seems to enhance and extend the aesthetic potential of the artifacts,” explains Dr. Kirk about the relationship between interactivity and aesthetics. As for the rug that reacts to noises, it’s called Lither in the Decoraction catalogue, which explains that it “responds to ambient sounds, specifically high pitches of loud voices or noises … deforms as a whole … then relaxes, leaving behind small parts that are kept deformed.” The rug’s ability to react to noise is achieved through the use of shape memory alloy wires sewn underneath it. These thread-like wires correspond to degrees of sound pitch. In this example, the rug is programmed to respond to noise through high pitch detected by a sound sensor, causing the wires to heat and deform, while a lack of noise causes the wires to relax and the rug to slowly return to its original shape. The possibility that Despot, Waterfall, and Lither can support self-reflection and self-care touches on another aspect of Dr. Nabil’s research, which is present throughout the Decoraction catalogue, but is also one that she is researching at Queen’s University: “What would all of this seamless interactivity with everyday objects be useful for?”

Since she was young, growing up in Cairo, Egypt, Dr. Nabil has asked herself, “Can decorative objects do more than simply play an aesthetic role?”

“Since I was a child, I wanted to be a designer. I ended up studying computing instead,” she says with just a hint of regret, noting that both her parents are architects and that they discouraged her from the field, claiming it “wasn’t the thing now.” Instead, she took up computer science, eventually working with various tech companies.

“I loved coding, but there was something empty in my heart. I was considered by my peers to be successful, but I was not happy. I always loved the design of decorative things. At some point I wanted to bring my two selves together – my designer and my developer selves.”

Dr. Nabil eventually formed a PhD proposal to blend technology with the physical environment of interior design, which was accepted and fully funded by Newcastle University. This produced Decoraction.

“People see decorative objects as beautiful things… I see much more than that. They support our well-being; they express who we are and our identity and our personalities; they tell stories and shape memories; they work for the remembrance of loved ones,” she explains.

This may have been especially acute during COVID-19, when the planet was confined during lockdowns. In these environments, without social contact, she says people had to rely on the objects around them for emotional support.

Dr. Kirk expands on that idea, saying, “As a species, humans love things. We fill our lives and our homes with them. We have deep and meaningful relationships to the things we own. Why would we not want to hybridize them, making them part physical and part digital? They could offer so much more with a layer of digital interactivity woven through them, enhancing our relationship to them, personalizing them further, giving them memory and the dynamic ability to respond to us. There is a real possibility of being able to instil a little bit of everyday magic into the objects around us – a little bit of enchantment makes the world a better place.”

As Decoraction demonstrates, household objects can do more – should do more – and not just for those living in a household, either. Dr. Nabil’s research includes designing with marginalized groups in order to assess how objects, innovations, and even fabrication technologies (think 3D printers, laser scanners, etc.) might impact them and their community.

“A lot of our work explores empowering underrepresented people and communities,” she explains. “Diversity and inclusion is required for this process to bring to our team new members who have a widened perspective. … Those with different identities and backgrounds.”

Presently, Dr. Nabil and her team are exploring advanced techniques for everyday embodied interaction thanks to a Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery grant received in 2021, in addition to a generous Discovery Launch Supplement grant.

The university was especially helpful in acquiring this grant, says Dr. Nabil, adding that Queen’s is very supportive in helping faculty apply for funding grants from federal and provincial governments.

“These grants… require different kinds of writing,” she explains, citing the differences between writing grant applications for NSERC compared with those for the Social Sciences and Humanities Research Council (SSHRC). “Otherwise, I would not have been able to get all of those funding applications approved,” she says.

One application of this research could be related to museums, says Dr. Nabil. Currently, the focus is on everyday objects, particularly in relation to inclusive communities – that is communities that provide equal opportunities for everyone.

Her latest research draws on previous work with people with visual impairment, who have been left out of the conversation regarding handheld devices and security.

A solution? Bendable phones.

“Instead of typing in a password, what if you could bend the edges of your phone in a certain pattern and that could be your PIN?” she asks.

Working with 16 participants with visual impairment, the team created and tested a prototype called BendyPass. Roughly the size of an iPhone, it’s made from silicone and has five sensors. Users create passwords by bending or folding the four corners, and/or pushing a button in the centre of the BendyPass. The team found that not only were bendy passwords as easy to input and as secure as regular PINs, but they were faster to put in.

While Dr. Nabil may produce possible solutions to problems, that’s not her end goal.

“Most of my projects involve experimenting with things or exploring new ideas. It’s not necessarily presenting end products or solutions, but opening up design spaces for the design community,” she explains, like her work with people with visual impairment, a community often underrepresented in today’s technological marketplace.

She gives the example of a student of hers, an immigrant to North America, who told her that Alexa – virtual assistant technology that answers questions through voice recognition – could not understand him or his wife, only his children. In this example, the immigrant parents have had to adapt to the technology, not the other way around. She points out that this example illustrates that the relationship between humans and technology is currently tipped in favour of the technology.

“I’ve always believed that technology should be able to change to adapt to us … I shouldn’t be changing my accent so that Alexa can understand what I’m saying,” says Dr. Nabil, who says her team aims to “empower people” by looking at physical objects in a different, more inclusive way.

“[Technology] should be designed to support people’s well-being, their self-care, their self-reflection, self-connection, and social engagement; to feel good about themselves, not focus-demanding, not intrusive. The current technology that helps people to stay connected actually causes them stress. What if something’s more touching than alarming, more emotional than intrusive?” she asks.

It’s another question in a series of questions – but it’s these questions that propel Dr. Nabil and her team as they explore the relationship between humans and technology, between us and everyday objects. In doing so, she asks that we demand more from these relationships, and wonders what possibilities and potential may be revealed if we do so.