Textination Newsline

Adaptive smart glove from MIT CSAIL researchers can send tactile feedback to teach users new skills, guide robots with more precise manipulation, and help train surgeons and pilots.

You’ve likely met someone who identifies as a visual or auditory learner, but others absorb knowledge through a different modality: touch. Being able to understand tactile interactions is especially important for tasks such as learning delicate surgeries and playing musical instruments, but unlike video and audio, touch is difficult to record and transfer.

To tap into this challenge, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and elsewhere developed an embroidered smart glove that can capture, reproduce, and relay touch-based instructions. To complement the wearable device, the team also developed a simple machine-learning agent that adapts to how different users react to tactile feedback, optimizing their experience. The new system could potentially help teach people physical skills, improve responsive robot teleoperation, and assist with training in virtual reality.

Will I be able to play the piano?
To create their smart glove, the researchers used a digital embroidery machine to seamlessly embed tactile sensors and haptic actuators (a device that provides touch-based feedback) into textiles. This technology is present in smartphones, where haptic responses are triggered by tapping on the touch screen. For example, if you press down on an iPhone app, you’ll feel a slight vibration coming from that specific part of your screen. In the same way, the new adaptive wearable sends feedback to different parts of your hand to indicate optimal motions to execute different skills.

The smart glove could teach users how to play the piano, for instance. In a demonstration, an expert was tasked with recording a simple tune over a section of keys, using the smart glove to capture the sequence by which they pressed their fingers to the keyboard. Then, a machine-learning agent converted that sequence into haptic feedback, which was then fed into the students’ gloves to follow as instructions. With their hands hovering over that same section, actuators vibrated on the fingers corresponding to the keys below. The pipeline optimizes these directions for each user, accounting for the subjective nature of touch interactions.

“Humans engage in a wide variety of tasks by constantly interacting with the world around them,” says Yiyue Luo MS ’20, lead author of the paper, PhD student in MIT’s Department of Electrical Engineering and Computer Science (EECS), and CSAIL affiliate. “We don’t usually share these physical interactions with others. Instead, we often learn by observing their movements, like with piano-playing and dance routines.

“The main challenge in relaying tactile interactions is that everyone perceives haptic feedback differently,” adds Luo. “This roadblock inspired us to develop a machine-learning agent that learns to generate adaptive haptics for individuals’ gloves, introducing them to a more hands-on approach to learning optimal motion.”

The wearable system is customized to fit the specifications of a user’s hand via a digital fabrication method. A computer produces a cutout based on individuals’ hand measurements, then an embroidery machine stitches the sensors and haptics in. Within 10 minutes, the soft, fabric-based wearable is ready to wear. Initially trained on 12 users’ haptic responses, its adaptive machine-learning model only needs 15 seconds of new user data to personalize feedback.

In two other experiments, tactile directions with time-sensitive feedback were transferred to users sporting the gloves while playing laptop games. In a rhythm game, the players learned to follow a narrow, winding path to bump into a goal area, and in a racing game, drivers collected coins and maintained the balance of their vehicle on their way to the finish line. Luo’s team found that participants earned the highest game scores through optimized haptics, as opposed to without haptics and with unoptimized haptics.

“This work is the first step to building personalized AI agents that continuously capture data about the user and the environment,” says senior author Wojciech Matusik, MIT professor of electrical engineering and computer science and head of the Computational Design and Fabrication Group within CSAIL. “These agents then assist them in performing complex tasks, learning new skills, and promoting better behaviors.”

Bringing a lifelike experience to electronic settings
In robotic teleoperation, the researchers found that their gloves could transfer force sensations to robotic arms, helping them complete more delicate grasping tasks. “It’s kind of like trying to teach a robot to behave like a human,” says Luo. In one instance, the MIT team used human teleoperators to teach a robot how to secure different types of bread without deforming them. By teaching optimal grasping, humans could precisely control the robotic systems in environments like manufacturing, where these machines could collaborate more safely and effectively with their operators.

“The technology powering the embroidered smart glove is an important innovation for robots,” says Daniela Rus, the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science at MIT, CSAIL director, and author on the paper. “With its ability to capture tactile interactions at high resolution, akin to human skin, this sensor enables robots to perceive the world through touch. The seamless integration of tactile sensors into textiles bridges the divide between physical actions and digital feedback, offering vast potential in responsive robot teleoperation and immersive virtual reality training.”

Likewise, the interface could create more immersive experiences in virtual reality. Wearing smart gloves would add tactile sensations to digital environments in video games, where gamers could feel around their surroundings to avoid obstacles. Additionally, the interface would provide a more personalized and touch-based experience in virtual training courses used by surgeons, firefighters, and pilots, where precision is paramount.

While these wearables could provide a more hands-on experience for users, Luo and her group believe they could extend their wearable technology beyond fingers. With stronger haptic feedback, the interfaces could guide feet, hips, and other body parts less sensitive than hands.

Luo also noted that with a more complex artificial intelligence agent, her team's technology could assist with more involved tasks, like manipulating clay or driving an airplane. Currently, the interface can only assist with simple motions like pressing a key or gripping an object. In the future, the MIT system could incorporate more user data and fabricate more conformal and tight wearables to better account for how hand movements impact haptic perceptions.

Luo, Matusik, and Rus authored the paper with EECS Microsystems Technology Laboratories Director and Professor Tomás Palacios; CSAIL members Chao Liu, Young Joong Lee, Joseph DelPreto, Michael Foshey, and professor and principal investigator Antonio Torralba; Kiu Wu of LightSpeed Studios; and Yunzhu Li of the University of Illinois at Urbana-Champaign.

The work was supported, in part, by an MIT Schwarzman College of Computing Fellowship via Google and a GIST-MIT Research Collaboration grant, with additional help from Wistron, Toyota Research Institute, and Ericsson.

Transitioning away from per- and polyfluoroalkyl substances (PFAS), which offer water- and oil-repelling properties on the outer shells of firefighter turnout gear, could bring potential performance tradeoffs, according to a new study from North Carolina State University.

The study showed that turnout gear without PFAS outer shell coatings were not oil-repellent, posing a potential flammability hazard to firefighters if exposed to oil and flame, said Bryan Ormond, assistant professor of textile engineering, chemistry and science at NC State and corresponding author of a paper describing the research.

“All oil repellents can also repel water, but all water repellents don’t necessarily repel oil,” Ormond said. “Diesel fuel is really difficult to repel, as is hydraulic fluid; in our testing, PFAS-treated materials repel both. In our tests, turnout gear without PFAS repelled water but not oil or hydraulic fluid.

“Further, oils seem to spread out even more on the PFAS-free gear, potentially increasing the hazard.”

PFAS chemicals – known as forever chemicals because of their environmental persistence – are used in food packaging, cookware and cosmetics, among other uses, but have recently been implicated in higher risks of cancer, higher cholesterol levels and compromised immune systems in humans. In response, firefighters have sought alternative chemical compounds – like the hydrocarbon wax coating used in the study – on turnout gear to repel water and oils.

Besides testing the oil- and water-repelling properties of PFAS-treated and PFAS-free outer garments, the NC State researchers also compared how the outer shells aged in job-related exposures like weathering, high heat and repeated laundering, and whether the garments remained durable and withstood tears and rips.

The study showed that PFAS-treated and PFAS-free outer shells performed similarly after exposure to UV rays and various levels of heat and moisture, as well as passes through heating equipment – similar to a pizza oven – and through washing machines.

“Laundering the gear is actually very damaging to turnout gear because of the washing machine’s agitation and cleaning agents used,” Ormond said.

“We also performed chemical analyses to see what’s happening during the weathering process,” said Nur Mazumder, an NC State doctoral student in fiber and polymer science and lead author of the paper. “Are we losing the PFAS chemistries, the PFAS-free chemistries or both when we age the garments? It turns out that we lost significant amounts of both of these finishes after the aging tests.”

Both types of garments performed similarly when tested for strength against tearing the outer shell fabric. The researchers say the PFAS and PFAS-free coatings didn’t seem to affect this attribute.

Ormond said that future work will explore how much oil repellency is needed by firefighters out in the field.
“Even with PFAS treatment, you see a difference between a splash of fluid and soaked-in fluid,” Ormond said. “For all of its benefits, PFAS-treated gear, when soaked, is dangerous to firefighters. So we need to really ask ‘What do firefighters need?’ If you’re not experiencing this need for oil repellency, there’s no worry about switching to non-PFAS gear. But firefighters need to know the non-PFAS gear will absorb oil, regardless of what those oils are.”

Andrew Hall, another NC State doctoral student in fiber and polymer science and co-author on the paper, is also testing dermal absorption, or taking the aged outer shell materials and placing them on a skin surrogate for a day or two. Are outer shell chemicals absorbed in the skin surrogate after these admittedly extreme exposure durations?

“Firefighting as a job is classified as a carcinogen but it shouldn’t have to be,” Ormond said. “How do we make better gear for them? How do we come up with better finishes and strategies for them?

“These aren’t just fabrics,” Ormond said. “They are highly engineered pieces of material that aren’t easily replaced.”

The paper appears in the Journal of Industrial Textiles. Funding for the research came from the Federal Emergency Management Agency’s Assistance to Firefighters Grants Program.

A new chemical process developed by Empa turns cotton into a fire-resistant fabric, that nevertheless retains the skin-friendly properties of cotton.

Conventional flame retardant cotton textiles suffer from release of formaldehyde and are uncomfortable to wear. Empa scientists managed to circumvent this problem by creating a physically and chemically independent network of flame retardants inside the fibers. This approach retains the inherently positive properties of cotton fibers, which account for three-quarters of the world's demand for natural fibers in clothing and home textiles. Cotton is skin-friendly because it can absorb considerable amounts of water and maintain a favorable microclimate on the skin.

For firefighters and other emergency service personnel, protective clothing provides the most important barrier. For such purposes, cotton is mainly used as an inner textile layer that needs additional properties: For example, it must be fireproof or protect against biological contaminants. Nevertheless, it should not be hydrophobic, which would create an uncomfortable microclimate. These additional properties can be built into the cotton fibers by suitable chemical modifications.

Durability vs. toxicity
"Until now, it has always taken a compromise to make cotton fireproof," says Sabyasachi Gaan, a chemist and polymer expert who works at Empa's Advanced Fibers lab. Wash-durable flame retardant cotton in industry is produced by treating the fabric with flame retardants, which chemically links to the cellulose in the cotton. Currently, the textile industry has no other choice than to utilize formaldehyde-based chemicals – and formaldehyde is classified as a carcinogen. This has been an unsolved problem for decades. While formaldehyde-based flame retardant treatments are durable, they have additional drawbacks: The -OH groups of cellulose are chemically blocked, which considerably reduces the capability of cotton to absorb water, which results in an uncomfortable textile.

Gaan knows the chemistry of cotton fibers well and has spent many years at Empa developing flame retardants based on phosphorus chemistry that are already used in many industrial applications. Now he has succeeded in finding an elegant and easy way to anchor phosphorous in form of an independent network inside the cotton.

Independent network between cotton fibers
Gaan and his colleagues Rashid Nazir, Dambarudhar Parida and Joel Borgstädt utilized a tri-functional phosphorous compound (trivinylphosphine oxide), which has the capability of reacting only with specifically added molecules (nitrogen compounds like piperazin) to form its own network inside cotton. This makes the cotton permanently fire-resistant without blocking the favorable -OH groups. In addition, the physical phosphine oxide network also likes water. This flame retardant treatment does not include carcinogenic formaldehyde, which would endanger textile workers during textile manufacturing. The phosphine oxide networks, thus formed, does not wash out: After 50 launderings, 95 percent of the flame retardant network is still present in the fabric.

To render additional protective functionalities to the flame retardant cotton developed at Empa, the researchers also incorporated in situ generated silver nanoparticles inside the fabric. This works nicely in a one-step process together with generating the phosphine oxide networks. Silver nanoparticles provide the fiber with antimicrobial properties and survive 50 laundry cycles, too.

A high-tech solution from the pressure cooker
"We have used a simple approach to fix the phosphine oxide networks inside the cellulose," Gaan says. "For our lab experiments, we first treated the cotton with an aqueous solution of phosphorus and nitrogen compounds and then steamed it in a readily available pressure cooker to facilitate the crosslinking reaction of the phosphorus and the nitrogen molecules." The application process is compatible with equipment used in the textile industry. "Steaming textiles after dyeing, printing and finishing is a normal step in textile industry. So it doesn't require an additional investment to apply our process," states the Empa chemist.

Meanwhile, this newly developed phosphorus chemistry and its application is protected by a patent application. "Two important hurdles remain," Gaan says. "For future commercialization we need to find a suitable chemical manufacturer who can produce and supply trivinylphosphine oxide. In addition, trivinylphosphine oxide has to be REACH-registered in Europe."

Contact:
Dr. Sabyasachi Gaan
Advanced Fibers
Phone: +41 58 765 7611
sabyasachi.gaan@empa.ch
 
Contact:
Prof. Dr. Manfred Heuberger
Advanced Fibers
Phone: +41 58 765 7878
manfred.heuberger@empa.ch

A gel that releases drugs
The novel phosphorus chemistry can also be used to develop other materials, e.g. to make hydrogels that can release drugs upon changes in pH. Such gels could find application in treating wounds that heal slowly. In such wounds, the pH of the skin surface increases and the new phosphorus-based gels can be triggered to release medication or a dye that alerts doctors and nurses to the problem. Empa has also patented the production of such hydrogels.