The classical guitarist’s plucking techniques can be accurately classified using multimedia signal processing

27 May 2024

UP Diliman

In classical guitar performances, various plucking gestures can be employed to evoke different tonal qualities in the produced sound. To investigate these gestural parameters and tonal variations, we classify two main plucking techniques, namely the apoyando and the tirando, along with their corresponding variations, from the audio and the video of a classical guitar performance. From the video signal, we extract right-hand gestural parameters using a hand keypoint detection model which locates twenty-one landmarks corresponding to the joints of the right hand. We achieved an average classification accuracy of 92.5% after building a model from our data set containing the images at the plucking onsets. Moreover, the model can accurately classify even on the succeeding frames after the onset, and we can achieve comparable accuracy even under various camera perspectives provided that the data set fall under the same camera perspective. From the audio signal, we achieved an average accuracy of 88.61% by extracting relevant features from the audio signals. We believe that these models can potentially be used as a pedagogical tool in learning how to properly and correctly perform these various plucking techniques.

This research allows us to understand how the classical guitarist’s performance gestures relate with the produced tone. By analyzing and processing the audio and the video of a classical guitar performance, we can infer the specific technique employed by the guitarists using machine learning. This is particularly helpful for tone production pedagogy since instrumental techniques in producing a certain desired tone vary among classical guitarists due to their physiological differences. Consequently, classical guitarists resort to the experimentation method of finding the desired tone. By adopting an objective and quantitative approach through multimedia signal processing, we can supplement and support the pedagogues in providing feedback to their guitar students with regards to tone production.

Authors: Carl Timothy Tolentino and Franz De Leon (Electrical and Electronics Engineering Institute, University of the Philippines Diliman)

Read the full paper: https://ieeexplore.ieee.org/document/10376130

The classical guitarist’s plucking techniques can be accurately classified using multimedia signal processing

In classical guitar performances, various plucking gestures can be employed to evoke different tonal qualities in the produced sound. To investigate these gestural parameters and tonal variations, we classify two main plucking techniques, namely the apoyando and the tirando, along with their corresponding variations, from the audio and the video of a classical guitar performance. From the video signal, we extract right-hand gestural parameters using a hand keypoint detection model which locates twenty-one landmarks corresponding to the joints of the right hand. We achieved an average classification accuracy of 92.5% after building a model from our data set containing the images at the plucking onsets. Moreover, the model can accurately classify even on the succeeding frames after the onset, and we can achieve comparable accuracy even under various camera perspectives provided that the data set fall under the same camera perspective. From the audio signal, we achieved an average accuracy of 88.61% by extracting relevant features from the audio signals. We believe that these models can potentially be used as a pedagogical tool in learning how to properly and correctly perform these various plucking techniques.

This research allows us to understand how the classical guitarist’s performance gestures relate with the produced tone. By analyzing and processing the audio and the video of a classical guitar performance, we can infer the specific technique employed by the guitarists using machine learning. This is particularly helpful for tone production pedagogy since instrumental techniques in producing a certain desired tone vary among classical guitarists due to their physiological differences. Consequently, classical guitarists resort to the experimentation method of finding the desired tone. By adopting an objective and quantitative approach through multimedia signal processing, we can supplement and support the pedagogues in providing feedback to their guitar students with regards to tone production.

Authors: Carl Timothy Tolentino and Franz De Leon (Electrical and Electronics Engineering Institute, University of the Philippines Diliman)

Read the full paper: https://ieeexplore.ieee.org/document/10376130