Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Asmaa Osman; Mohamed Shalaby; Mona Soliman; Khaled Elsayed

doi:10.34028/iajit

Article Details

Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Authors:

Asmaa Osman | Mohamed Shalaby | Mona Soliman | Khaled Elsayed

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.21 Issue 3, pp.458-465

Document Type:

Article

Publication Date:

1/2024

Document Language:

English

Document ID (Org No.):

2001389730

pISSN:

2309-4524

DOI:

10.34028/iajit

Keywords:

Abstract:

 Image captioning is a major artificial intelligence research field that involves visual interpretation and linguistic description of a corresponding image. Successful image captioning relies on acquiring as much information as feasible from the original image. One of these essential bits of knowledge is the topic or the concept that the image is associated with. Recently, concept modeling technique has been utilized in English image captioning for completely capturing the image contexts and make use of these contexts to produce more accurate image descriptions. In this paper, a concept-based model is proposed for Arabic Image Captioning (AIC). A novel Vision-based Multi-Encoder Transformer Architecture (ViMETA) is proposed for handling the multi-outputs result from the concept modeling technique while producing the image caption. BiLingual Evaluation Understudy (BLEU) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) standard metrics have been used to evaluate the proposed model using the Flickr8K dataset with Arabic captions. Furthermore, qualitative analysis has been conducted to compare the produced captions of the proposed model with the ground truth descriptions. Based on the experimental results, the proposed model outperformed the related works both quantitatively, using BLEU and ROUGE metrics, and qualitatively.

Article Details

Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Authors:

Asmaa Osman | Mohamed Shalaby | Mona Soliman | Khaled Elsayed

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.21 Issue 3, pp.458-465

Document Type:

Article

Publication Date:

1/2024

Document Language:

English

Document ID (Org No.):

2001389730

pISSN:

2309-4524

DOI:

10.34028/iajit

Arabic image captioning

transformer

concept

computer vision

Issues

Article Details

Ar-CM-ViMETA: Arabic Image Captioning based on Concept Model and Vision-based Multi-Encoder Transformer Architecture

Authors: Asmaa Osman | Mohamed Shalaby | Mona Soliman | Khaled Elsayed

Publisher: Zarqa University, Jordan

Journal: The International Arab Journal of Information Technology (IAJIT) Vol.21 Issue 3, pp.458-465

Document Type: Article

Publication Date: 1/2024

Document Language: English

Document ID (Org No.): 2001389730

pISSN: 2309-4524

DOI: 10.34028/iajit

Arabic image captioning

transformer

concept

computer vision

Issues

Authors:

Asmaa Osman | Mohamed Shalaby | Mona Soliman | Khaled Elsayed

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.21 Issue 3, pp.458-465

Document Type:

Article

Publication Date:

1/2024

Document Language:

English

Document ID (Org No.):

2001389730

pISSN:

2309-4524

DOI:

10.34028/iajit