Internet of Things in the audio domain
The Internet of Things (IoT) enables a wide range of devices to talk to each other. It is expected that by 2030, 25 billion everyday objects will be connected to the IoT. Over the last couple of years, IoT concepts have also started to spread into the music, audio, and multimedia domain.
The Internet of Musical Things (IoMusT) is mostly focused on musical applications (concerts, music production, music pedagogy, etc.) that are consumed by humans. Developing convincing applications that are accepted by users depends on a high quality of service (QoS) and a high quality of experience (QoE). In other words, for a convincing user experience, the audio quality has to be high and without perceivable packet loss, and the end-to-end latency has to be imperceptibly low when performing remotely together. Research in the AudioLabs will have strong connections to the Audio and Media Technologies division of Fraunhofer IIS to benefit from its many years of experience in low-delay and high-quality audio coding.
The Internet of Audio Things (IoAuT) focuses on the more general field of audio applications, such as for industrial applications. The audio-related data captured and generated in IoAuT networks is not necessarily consumed by humans, but rather by machines. For example, a distributed IoAuT network along a busy highway could jointly measure and analyze traffic noise to detect traffic jams and car accidents in real time. An interesting research question for that domain is “How can we extract meaningful information from a sound field?” This is the overlap to the research field of Prof. Meinard Müller, “Semantic Audio Processing.” With the help of sound field analysis – part of the Spatial Audio Processing research of Prof. Emanuël Habets – questions like “How to determine the location or directivity of sound sources” can be answered.
In another use case, an IoAuT network could capture a sound field at different spatial locations simultaneously and intelligently process those recordings for further uses, such as for sound field navigation in VR applications or to create privacy zones in an existing sound field. A strong connection to the VR Research field of Prof. Frank Wefers is a natural fit here.
When it comes to sending audio between IoT nodes in a compressed way, algorithms for audio data reduction by means of perceptual audio coding are needed; here, close cooperation with the Audio Coding research field of Prof. Jürgen Herre is apparent.
Besides the typical AudioLabs use cases mentioned above, there are also some that might not be that obvious: ecoacoustics, for example, looks at the ecological role of sounds. Or even more surprising: applications in agriculture, to detect illegal tree felling for example.
Despite all the possibilities, there are also some current challenges in the field of Audio IoT. There are technical constraints such as the battery power of mobile devices, limited computational capabilities, wireless connectivity constraints (bandwidth, range), and latency. But the most significant barriers are concerns regarding data privacy, security, and the interoperability of IoT devices.
As you can see, the possibilities are very diverse, and the tasks are certainly challenging within the “Internet of Things in the audio domain” research field of Prof. Nils Peters.