TIGP (SNHCC) -- Use of Artificial Intelligence in Music and its Broad Impacts
- 講者Kristen Yeon-Ji Yun 教授 (普渡大學)
邀請人：TIGP (SNHCC) - 時間2023-12-11 (Mon.) 15:00 ~ 16:00
- 地點資訊所新館106演講廳
摘要
Artificial Intelligence has demonstrated impressive progress in music. Programs like AIVA can compose new music when preferred style and genre are entered. AI is widely employed for recording music as well. However, AI has not been used much in music performance and education. This session will introduce using AI in those areas and its broad impact.
1) AI can generate real-time images/videos while it listens to music. The technology analyzes musical elements and creates feature matrices. Through the compiled feature matrices, the trained neural network will predict the genre, and emotional systems. From the predicted information, the neural networks generate text prompts to generative models to render images and concatenate the output frames together and make a video. This technology is expected to help hearing impaired people appreciate music through visual images.
2) AI technology can be embedded to create supporting tools for music performers. With integrating techniques from computer vision, natural language processing, and audio analysis, AI can analyze a musician's sound and compare it to digitized music scores to detect deviations in intonation, rhythm, and dynamics. It can compare the captured video of performers to a database of samples recorded with correct postures, allowing it to recommend better postures, which can both improve musical performance and reduce injury risks. AI can also play the part of one or several instruments to replace absent musicians, matching tempo, and style of the human musicians through audio analysis of their performance while also responding in real-time to verbal instructions.
1) AI can generate real-time images/videos while it listens to music. The technology analyzes musical elements and creates feature matrices. Through the compiled feature matrices, the trained neural network will predict the genre, and emotional systems. From the predicted information, the neural networks generate text prompts to generative models to render images and concatenate the output frames together and make a video. This technology is expected to help hearing impaired people appreciate music through visual images.
2) AI technology can be embedded to create supporting tools for music performers. With integrating techniques from computer vision, natural language processing, and audio analysis, AI can analyze a musician's sound and compare it to digitized music scores to detect deviations in intonation, rhythm, and dynamics. It can compare the captured video of performers to a database of samples recorded with correct postures, allowing it to recommend better postures, which can both improve musical performance and reduce injury risks. AI can also play the part of one or several instruments to replace absent musicians, matching tempo, and style of the human musicians through audio analysis of their performance while also responding in real-time to verbal instructions.
BIO
Kristen Yeon-Ji Yun, a Clinical Associate Professor in the Music Department at Purdue University, is active as a soloist, chamber musician, musical scholar, and clinician. She has toured many countries including Malaysia, Thailand, Korea, Mexico, Spain, France, Germany, Taiwan, etc. giving a series of successful concerts and master classes. Yun performs on a French cello, made by Guersan in 1766. In September 2023, she received a grant from the National Science Foundation as the principal investigator on the topic “Artificial Intelligence Technology for Future Music Performers”.