IDEAS home Printed from https://ideas.repec.org/a/igg/jswis0/v19y2023i1p1-20.html
   My bibliography  Save this article

Query-Guided Refinement and Dynamic Spans Network for Video Highlight Detection and Temporal Grounding in Online Information Systems

Author

Listed:
  • Yifang Xu

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Yunzhuo Sun

    (School of Physics and Electronics, Hubei Normal University, China)

  • Zien Xie

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Benxiang Zhai

    (School of Electronic Science and Engineering, Nanjing University, China)

  • Youyao Jia

    (Gosuncn Chuanglian Technology Co., Ltd., Guangzhou, China)

  • Sidan Du

    (School of Electronic Science and Engineering, Nanjing University, China)

Abstract

With the surge in online video content, finding highlights and key video segments have garnered widespread attention. Given a textual query, video highlight detection (HD) and temporal grounding (TG) aim to predict frame-wise saliency scores from a video while concurrently locating all relevant spans. Despite recent progress in DETR-based works, these methods crudely fuse different inputs in the encoder, which limits effective cross-modal interaction. To solve this challenge, the authors design QD-Net (query-guided refinement and dynamic spans network) tailored for HD&TG. Specifically, they propose a query-guided refinement module to decouple the feature encoding from the interaction process. Furthermore, they present a dynamic span decoder that leverages learnable 2D spans as decoder queries, which accelerates training convergence for TG. On QVHighlights dataset, the proposed QD-Net achieves 61.87 HD-HIT@1 and 61.88 TG-mAP@0.5, yielding a significant improvement of +1.88 and +8.05, respectively, compared to the state-of-the-art method.

Suggested Citation

  • Yifang Xu & Yunzhuo Sun & Zien Xie & Benxiang Zhai & Youyao Jia & Sidan Du, 2023. "Query-Guided Refinement and Dynamic Spans Network for Video Highlight Detection and Temporal Grounding in Online Information Systems," International Journal on Semantic Web and Information Systems (IJSWIS), IGI Global, vol. 19(1), pages 1-20, January.
    • Handle: RePEc:igg:jswis0:v:19:y:2023:i:1:p:1-20
    as

    Download full text from publisher

    File URL: http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJSWIS.332768
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lee, Jung Hoon & Hancock, Marguerite Gong & Hu, Mei-Chih, 2014. "Towards an effective framework for building smart cities: Lessons from Seoul and San Francisco," Technological Forecasting and Social Change, Elsevier, vol. 89(C), pages 80-99.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vu, Khuong & Hartley, Kris, 2018. "Promoting smart cities in developing countries: Policy insights from Vietnam," Telecommunications Policy, Elsevier, vol. 42(10), pages 845-859.
    2. Jiabin Liu & Ji Han, 2017. "Does a Certain Rule Exist in the Long-Term Change of a City’s Livability? Evidence from New York, Tokyo, and Shanghai," Sustainability, MDPI, vol. 9(10), pages 1-15, September.
    3. Sabina Baraniewicz-Kotasińska, 2022. "The Scandinavian Third Way as a Proposal for Sustainable Smart City Development—A Case Study of Aarhus City," Sustainability, MDPI, vol. 14(6), pages 1-24, March.
    4. Kumar, Harish & Singh, Manoj Kumar & Gupta, M.P. & Madaan, Jitendra, 2020. "Moving towards smart cities: Solutions that lead to the Smart City Transformation Framework," Technological Forecasting and Social Change, Elsevier, vol. 153(C).
    5. Łukasz Brzeziński & Magdalena Krystyna Wyrwicka, 2022. "Fundamental Directions of the Development of the Smart Cities Concept and Solutions in Poland," Energies, MDPI, vol. 15(21), pages 1-52, November.
    6. Kajikawa, Yuya & Mejia, Cristian & Wu, Mengjia & Zhang, Yi, 2022. "Academic landscape of Technological Forecasting and Social Change through citation network and topic analyses," Technological Forecasting and Social Change, Elsevier, vol. 182(C).
    7. Jaewon Lim & Jae Hong Kim, 2019. "Joint Determination of Residential Relocation and Commuting: A Forecasting Experiment for Sustainable Land Use and Transportation Planning," Sustainability, MDPI, vol. 11(1), pages 1-24, January.
    8. Renata Biadacz & Marek Biadacz, 2021. "Implementation of “Smart” Solutions and An Attempt to Measure Them: A Case Study of Czestochowa, Poland," Energies, MDPI, vol. 14(18), pages 1-28, September.
    9. Margarida Rodrigues & Mário Franco, 2018. "Measuring the Performance in Creative Cities: Proposal of a Multidimensional Model," Sustainability, MDPI, vol. 10(11), pages 1-21, November.
    10. Joel Serey & Luis Quezada & Miguel Alfaro & Guillermo Fuertes & Rodrigo Ternero & Gustavo Gatica & Sebastian Gutierrez & Manuel Vargas, 2020. "Methodological Proposals for the Development of Services in a Smart City: A Literature Review," Sustainability, MDPI, vol. 12(24), pages 1-28, December.
    11. Mona Treude, 2021. "Sustainable Smart City—Opening a Black Box," Sustainability, MDPI, vol. 13(2), pages 1-15, January.
    12. Habib M. Alshuwaikhat & Yusuf A. Adenle & Thamer Almuhaidib, 2022. "A Lifecycle-Based Smart Sustainable City Strategic Framework for Realizing Smart and Sustainability Initiatives in Riyadh City," Sustainability, MDPI, vol. 14(14), pages 1-17, July.
    13. Niusha Esmaeilpoorarabi & Tan Yigitcanlar & Mirko Guaralda, 2016. "Place quality and urban competitiveness symbiosis? A position paper," International Journal of Knowledge-Based Development, Inderscience Enterprises Ltd, vol. 7(1), pages 4-21.
    14. Bresciani, Stefano & Ferraris, Alberto & Del Giudice, Manlio, 2018. "The management of organizational ambidexterity through alliances in a new context of analysis: Internet of Things (IoT) smart city projects," Technological Forecasting and Social Change, Elsevier, vol. 136(C), pages 331-338.
    15. Abbate, Tindara & Cesaroni, Fabrizio & Cinici, Maria Cristina & Villari, Massimo, 2019. "Business models for developing smart cities. A fuzzy set qualitative comparative analysis of an IoT platform," Technological Forecasting and Social Change, Elsevier, vol. 142(C), pages 183-193.
    16. Małgorzata Baran & Monika Kłos & Monika Chodorek & Karolina Marchlewska-Patyk, 2022. "The Resilient Smart City Model–Proposal for Polish Cities," Energies, MDPI, vol. 15(5), pages 1-23, March.
    17. Seung-Min Jung & Sungwoo Park & Seung-Won Jung & Eenjun Hwang, 2020. "Monthly Electric Load Forecasting Using Transfer Learning for Smart Cities," Sustainability, MDPI, vol. 12(16), pages 1-20, August.
    18. van den Buuse, Daniel & Kolk, Ans, 2019. "An exploration of smart city approaches by international ICT firms," Technological Forecasting and Social Change, Elsevier, vol. 142(C), pages 220-234.
    19. Solis, Miriam & Bashar, Samira Binte, 2022. "Social equity implications of advanced water metering infrastructure," Utilities Policy, Elsevier, vol. 79(C).
    20. Nilssen, Maja, 2019. "To the smart city and beyond? Developing a typology of smart urban innovation," Technological Forecasting and Social Change, Elsevier, vol. 142(C), pages 98-104.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:igg:jswis0:v:19:y:2023:i:1:p:1-20. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Journal Editor (email available below). General contact details of provider: https://www.igi-global.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.