Medical Monitor with Eyes and Ears

In intensive care units (ICU), every second counts. In emergencies, doctors and nurses need to make the right decisions quickly. Fraunhofer researchers have developed a smart “proxemic monitor” which has optimized the processes in the most sensitive area of a hospital and clearly shows the data of the connected medical devices while avoiding false alarms. The screen can be controlled from a distance, without contact, by means of gestures and voice commands, thereby reducing the risk of transmitting pathogens.
 
The “proxemic monitor” is connected to a PC, which in turn is connected to the database in which all the vital data of the patients in intensive care converge.
 
Everything had just been quiet in the control room of the intensive care unit. Suddenly, though, there is excitement: the alarms on several medical devices in different rooms are sounding. Monitors are flashing and beeping. A drop in blood pressure, cardiac arrhythmia – the computer monitors show exactly what has to be done and where. Doctors and nurses rush to the rooms of the patients. Medical equipment there provides detailed information on the status of the critically ill patients. “It‘s not easy to keep track of everything here during hectic situations,” said Paul Chojecki, scientist from the “Vision & Imaging Technology” Department at the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut, HHI, in Berlin. 
 
Controlled by Contact-Free Gestures 
Together with his team as part of the joint project “control room,” the scientist has developed an intelligent monitor. It quickly shows doctors and nurses the most important information about the vital signs of their intensive care patients. The screen has interfaces to the medical equipment in the room, as well as with the information systems in the hospital, and can be controlled by contact-free gestures and voice commands. The graphics of its display adapt to the distance from which it is viewed. “From the door, the doctor sees the data in a correspondingly large size. As he gets closer, the screen displays detailed information,” Chojecki explains. A 3-D camera provides the necessary data. The user interface is programmed based upon the web, so it is also suitable for mobile devices, such as tablets. The scientists are presenting a demonstration of the “proxemic monitor” at CeBIT 2016 (Hall 6, Stand B36). A practical test in cooperation with Uniklinik RWTH Aachen is planned for later this year.
 
The system evaluates the data of the medical devices on the basis of the smart alarm design of the project partner, the Medical Engineering Department of Aachen University Hospital. This prevents false alarms. “Intensive care doctors have told us that this is a big problem. Current devices stick stubbornly to determined limits, without including all of the relevant factors that are necessary in order to comprehensively assess the risk situation. In addition, the acoustic signals produce a very loud noise, which is not beneficial to the health of either the staff or the patient,” Chojecki said.
 
Less Pathogens Transmitted 
Another advantage of the gesture control is that the doctor or health care provider does not have to touch the devices directly. “The transfer of pathogens in hospitals, particularly in intensive care units, is still a problem. The obligatory hand hygiene is sometimes forgotten, and viruses as well as bacteria are carried from room to room,” Chojecki described. The smart screen can be programmed at the bedside, as well as used in the control room (with a different configuration).
 
Chojecki added, “That‘s an important distinction, since it is not legally permitted for all of the instrument functions in the patients‘ rooms to be accessed from the control room. It has to be ensured that the doctor actually examines the patient before changing the treatment”.
 
Three different cameras and a microphone scan the area in front of the monitor. Using the video data, the built-in software of the HHI analyzes whether there are people in the room, how far away they are from the screen, and what movements they are making. Depending on the distance, the display and functionality of the monitor changes.
 
“Our monitor distinguishes between near, medium, and further distance. The cameras cover a maximum distance of four meters,” Chojecki explained. From the medium distance, the cursor can be controlled with arm movements, and commands or short reports can be input by voice. With pre-programmed gestures, for example, a video call can be started, in order to have discussions with other physicians within or outside of the hospital.
 
“We have given the monitor eyes and ears so as to allow for multi-modal interaction between the user and the system. Our software records distances and movements of the user in a contactless manner, interprets them, and converts them into commands for operating systems or machines,” Chojecki explained.