Neural Volumetric Representations for Real-Time 3D Scene Reconstruction Using Multi-Modal Learning Algorithm

Pidatala Devendrababu; Preeti Jha

doi:10.34028/iajit

Article Details

Neural Volumetric Representations for Real-Time 3D Scene Reconstruction Using Multi-Modal Learning Algorithm

Authors:

Pidatala Devendrababu | Preeti Jha

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.22 Issue 6, pp.1194-1202

Document Type:

Article

Publication Date:

1/2025

Document Language:

English

Document ID (Org No.):

2001389816

pISSN:

2309-4524

DOI:

10.34028/iajit

Keywords:

Abstract:

 Deep Learning (DL) is a subfield of Machine Learning (ML) models used in various complex fields. DL algorithms are mostly widely used to reconstruct 3D images collected from multiple online sources. It is a very challenging task for the existing algorithms to reconstruct 2D images into 3D pictures without losing high-quality pixels because of the complex scenes with different lighting situations, dynamic components, and occlusions. This paper presents a novel real-time 3D scene reconstruction using neural volumetric representations combined with a Multi-Modal Learning Algorithm (MMLA). The proposed MMLA focuses on solving issues like volumetric representations of scenes, which are improved by combining numerous modalities such as RGB images, depth sensors, and Inertial Measurement Unit (IMU) data. The MMLA combines the DeepVoxels model and Neural Radiance Fields (NeRF) model, which it calls the Neural Rendering technique, to learn complex patterns in 3D scenes. The pre-trained model EfficientNet accurately obtained the 3D- reconstruction patterns and understood the spatial structures that transfer to the proposed MMLA. The proposed MMLA performance is analyzed using the ShapeNet dataset, which consists of 2D images. Finally, the experimental results show that the proposed MMLA outperforms the superior performance in terms of Mean Squared Error (MSE) of 0.167, Root Mean Squared Error (RMSE) of 0.50, and Mean Absolute Error (MAE) of 1.1. These results may differ from other datasets.

Article Details

Neural Volumetric Representations for Real-Time 3D Scene Reconstruction Using Multi-Modal Learning Algorithm

Authors:

Pidatala Devendrababu | Preeti Jha

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.22 Issue 6, pp.1194-1202

Document Type:

Article

Publication Date:

1/2025

Document Language:

English

Document ID (Org No.):

2001389816

pISSN:

2309-4524

DOI:

10.34028/iajit

Deepvoxels

neural rendering technique

NeRF

MMLA

Issues

Article Details

Neural Volumetric Representations for Real-Time 3D Scene Reconstruction Using Multi-Modal Learning Algorithm

Authors: Pidatala Devendrababu | Preeti Jha

Publisher: Zarqa University, Jordan

Journal: The International Arab Journal of Information Technology (IAJIT) Vol.22 Issue 6, pp.1194-1202

Document Type: Article

Publication Date: 1/2025

Document Language: English

Document ID (Org No.): 2001389816

pISSN: 2309-4524

DOI: 10.34028/iajit

Deepvoxels

neural rendering technique

NeRF

MMLA

Issues

Authors:

Pidatala Devendrababu | Preeti Jha

Publisher:

Zarqa University, Jordan

Journal:

The International Arab Journal of Information Technology (IAJIT) Vol.22 Issue 6, pp.1194-1202

Document Type:

Article

Publication Date:

1/2025

Document Language:

English

Document ID (Org No.):

2001389816

pISSN:

2309-4524

DOI:

10.34028/iajit