IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v9y2021i15p1831-d607750.html
   My bibliography  Save this article

Style Transformation Method of Stage Background Images by Emotion Words of Lyrics

Author

Listed:
  • Hyewon Yoon

    (Department of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, Korea)

  • Shuyu Li

    (Department of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, Korea)

  • Yunsick Sung

    (Department of Multimedia Engineering, Dongguk University-Seoul, Seoul 04620, Korea)

Abstract

Recently, with the development of computer technology, deep learning has expanded to the field of art, which requires creativity, which is a unique ability of humans, and an understanding of the human emotions expressed in art to process them as data. The field of art is integrating with various industrial fields, among which artificial intelligence (AI) is being used in stage art, to create visual images. As it is difficult for a computer to process emotions expressed in songs as data, existing stage background images for song performances are human designed. Recently, research has been conducted to enable AI to design stage background images on behalf of humans. However, there is no research on reflecting emotions contained in song lyrics to stage background images. This paper proposes a style transformation method to reflect emotions in stage background images. First, multiple verses and choruses are derived from song lyrics, one at a time, and emotion words included in each verse and chorus are extracted. Second, the probability distribution of the emotion words is calculated for each verse and chorus, and the image with the most similar probability distribution from an image dataset with emotion word tags in advance is selected for each verse and chorus. Finally, for each verse and chorus, the stage background images with the transferred style are outputted. Through an experiment, the similarity between the stage background and the image transferred to the style of the image with similar emotion words probability distribution was 38%, and the similarity between the stage background image and the image transferred to the style of the image with completely different emotion word probability distribution was 8%. The proposed method reduced the total variation loss of change from 1.0777 to 0.1597. The total variation loss is the sum of content loss and style loss based on weights. This shows that the style transferred image is close to edge information about the content of the input image, and the style is close to the target style image.

Suggested Citation

  • Hyewon Yoon & Shuyu Li & Yunsick Sung, 2021. "Style Transformation Method of Stage Background Images by Emotion Words of Lyrics," Mathematics, MDPI, vol. 9(15), pages 1-20, August.
    • Handle: RePEc:gam:jmathe:v:9:y:2021:i:15:p:1831-:d:607750
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/9/15/1831/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7390/9/15/1831/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Shuyu Li & Sejun Jang & Yunsick Sung, 2019. "Automatic Melody Composition Using Enhanced GAN," Mathematics, MDPI, vol. 7(10), pages 1-13, September.
    2. Yunsick Sung & Yong Jin & Jeonghoon Kwak & Sang-Geol Lee & Kyungeun Cho, 2018. "Advanced Camera Image Cropping Approach for CNN-Based End-to-End Controls on Sustainable Computing," Sustainability, MDPI, vol. 10(3), pages 1-13, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shuyu Li & Yunsick Sung, 2023. "Transformer-Based Seq2Seq Model for Chord Progression Generation," Mathematics, MDPI, vol. 11(5), pages 1-14, February.
    2. Arulsamy, Karen & Delaney, Liam, 2022. "The impact of automatic enrolment on the mental health gap in pension participation: Evidence from the UK," Journal of Health Economics, Elsevier, vol. 86(C).
    3. Wenkai Huang & Feng Zhan, 2023. "A Novel Probabilistic Diffusion Model Based on the Weak Selection Mimicry Theory for the Generation of Hypnotic Songs," Mathematics, MDPI, vol. 11(15), pages 1-26, July.
    4. Shuyu Li & Yunsick Sung, 2021. "INCO-GAN: Variable-Length Music Generation Method Based on Inception Model-Based Conditional GAN," Mathematics, MDPI, vol. 9(4), pages 1-16, February.
    5. Shuyu Li & Yunsick Sung, 2023. "MRBERT: Pre-Training of Melody and Rhythm for Automatic Music Generation," Mathematics, MDPI, vol. 11(4), pages 1-14, February.
    6. Sejun Jang & Shuyu Li & Yunsick Sung, 2020. "FastText-Based Local Feature Visualization Algorithm for Merged Image-Based Malware Classification Framework for Cyber Security and Cyber Defense," Mathematics, MDPI, vol. 8(3), pages 1-13, March.
    7. Lvyang Qiu & Shuyu Li & Yunsick Sung, 2021. "DBTMPE: Deep Bidirectional Transformers-Based Masked Predictive Encoder Approach for Music Genre Classification," Mathematics, MDPI, vol. 9(5), pages 1-17, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:9:y:2021:i:15:p:1831-:d:607750. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.