Scoop from Eindhoven: Students Technical University present first drone which can express emotions
Student team Blue Jay from the Technical University of Eindhoven (TU/e) proudly present the first interactive drone which can express emotions. By doing this, it can fulfil an important function in saving human lives. During fire emergencies it is essential that no one panics and that an evacuation runs smooth and efficient. The drone of the Blue Jay team can autonomously search for elderly, subsequently calm them down and effectively guide them to safety. On July 9, the team will demonstrate how this works in practice.
Vulnerability of elderly in nursing homes
A nursing home is a place full of emotions; from joy because of a full bingo card to sadness due to the loss of a loved one. But in some instances, there is fear and panic. 600 times a year, a nursing home in the Netherlands needs to be evacuated due to a fire hazard. During an evacuation, elderly often do not know what to do, leading them to be afraid or to panic. Due to this, they are often not able to find their way towards safety. This, potentially in combination with a shortage of nurses, can have serious consequences, in the worst case even death. In 2020 alone 25 Dutch people who are not fully self-reliant have died due to situations like these. To prevent that, Blue Jay is developing the first drone which can calm elderly down and which can actively help them with finding the emergency exit.
Recognizing and expressing emotions
There is a small camera on the drone, which has been programmed to see and recognize the emotions from elderly. If a resident looks happy, the drone will recognize that as a happy person. If a person changes his/her expression to surprised, the drone will recognize that as a surprised person. Once an emotion has been registered, the drone will respond by showing an emotion itself. This happens using a couple of eyes on the front of the drone. To make the drone seem natural it is essential that a natural reaction follows the emotion of the resident. For example, if a resident looks happy, it is natural that the drone looks back happy and not angry or sad.
To let the drone express the correct emotions, the Blue Jay team collaborated with a group of elderly from the nursing home Joris Zorg. Via multiple trials in this nursing home, it has been investigated how residents and employees respond to the presence and expressed emotions of the drone. Manon Schreurs, health and safety advisor at Joris Zorg: “We are proud to contribute by investigating the human aspect, in collaboration with our residents.” Rik Schutte, Lead Interaction Design at Blue Jay, is also enthusiastic about the collaboration between the parties: “This enables us to receive direct feedback from the elderly in nursing homes.”
Enthusiasm for now and for the future
Not only the residents of the nursing home, but also the fire brigade is enthusiastic. Paul van Dooren, Advisor Innovation at Brandweer Brabant-Zuidoost: “The interaction between human being and drone creates an added value to the existing resources. Resources like fire alarms do give a signal, but they cannot provide individual help. This drone is a good step in the direction of being able to provide this specific support. Hopefully, this application can be fine-tuned in the future, for example by using speech- or object detection for the purpose of generating a full analysis of the situation.”
Interview Internship
Currently, the school semester is running to an end. This means that three of our interns are finalizing their reports. To show you what an internship looks like at Blue Jay Eindhoven, we had an interview with Joshua!
What is your name and what are you currently studying?
My name is Joshua de Leeuw and I am currently studying Mechatronics. I am in my 3rd year, so I am doing a practical (interim) internship at Blue Jay.
What exactly are you doing for your internship?
I am working on drone-to-drone (D-2-D) and drone-to-device (D-2-C) communication. This entails making sure that drones can wirelessly communicate with other electronics. For example, D-2-C communication can be used to open doors or to send emergency signals. D-2-D signals can be used by drones to send locations to each other. By doing that, multiple drones can work together and can more quickly search through the building for people.
What did your internship look like?
In the first weeks, I worked on my research report, investigating in which methods drones can communicate wirelessly. Based on that, I selected two potential method (433 MHz UHF-bands and Wi-Fi) After that, I worked on a UML diagram to illustrate how all programs should work together. Once this worked, I started programming in Python to set up the 433 MHz communication. During this programming, I got in touch with companies, such as NXP. I got advised to further investigate Wi-Fi, especially IEEE 802.11p. Due to this, I also set up this form of communication in Python.
After both forms of communication were set up, I started testing them both. This testing was based on factors such as communication range (in meters), resistance to noise and speed of computation.
Based on these tests, I determined that IEEE 802.11p is the best method to communicate. I received a VERA module from NXP, which will eventually be put on the drone.
Currently, I am almost finished with my report. I am checking everything on spelling and grammar, after which the report will be handed in.
Why did you decide to join Blue Jay for your internship?
I could choose between Blue Jay and another project-based internship (at a non-student team). Eventually, I chose Blue Jay, because:
- I can make all project decisions by myself. There are limitations in money and the decisions must fit the goal of Blue Jay. Aside of that, everything was completely up to me. In other places, you often do not get this much responsibility or independence.
- I wanted to experience working with fellow students. Aside of the work, there is also lots of fun and ‘gezelligheid’. In other companies, the atmosphere can be formal and hierarchical, while this is not the case at Blue Jay.
- I wanted to learn how drones fly and work.
What are the lesser points of working at Blue Jay?
- Currently, a problem is Covid-19. Although I can go to the office, it is not the same as a normal situation. The fun aspect is lower since we are not allowed to do any fun activities.
- Everyone is a student and thus less experienced in their work. This means that for some aspects you are reliant on sponsors. Although it is easy to get in contact with them, it does cost extra time and effort.
How did you get in touch with us?
There were several moments when I got in touch. The first one is the Fontys Meet and Match, during which I had a nice chat with old team members. The next moment was an emotion detection test on the TU/e campus in which I was a participant. The last touch point was the list of potential internship companies, on which you were present as well. During all these moments, I got interested in joining and thus I phoned the previous team manager Anne. Then, I arranged an interview and was allowed to join the team.
Are you also interested in joining Blue Jay (for your internship)? You can send an email to info@bluejayeindhoven.nl or visit www.bluejayeindhoven.nl.