Creating a Clinical Psychology Dataset with Synthetic Data: Automatic Detection of Cognitive Distortions Classified with NLP

Hakkı Halil Babacan; Ramazan Oğuz; Yahya Kemal Beyitoğlu

Article Details

Creating a Clinical Psychology Dataset with Synthetic Data: Automatic Detection of Cognitive Distortions Classified with NLP

Authors:

Hakkı Halil Babacan | Ramazan Oğuz | Yahya Kemal Beyitoğlu

Publisher:

Fırat University

Main Topics:

Doğal Dil İşleme

Journal:

Fırat Üniversitesi Mühendislik Bilimleri Dergisi Vol.37 Issue 1, pp.83-92

Document Type:

Article

Publication Date:

3/2025

Document Language:

English

Document ID (Org No.):

2001334856

pISSN:

1308-9072

Keywords:

Abstract:

 Cognitive distortions are thought errors that lead individuals to perceive reality in a misleading way and are strongly associated with psychopathologies. Therefore, accurately identifying and classifying distortions can enhance the effectiveness of cognitive-behavioral therapy CBT. This study investigates the effectiveness of deep learning and NLP techniques for the automatic detection of cognitive distortions. The RoBERTa model was trained using English synthetic data generated by GPT-4 2000 examples and the dataset from Shreevastava and Foltz 1590 cognitive distortion examples, 933 non-distortion examples. Three scenarios were tested: the original dataset, the synthetic dataset, and their combination. The results showed that synthetic data is a strong resource. Accuracy rates were 60.67 original, 94.51 synthetic, and 77.18 combined. The GPT-4-based dataset provided almost perfect F1 scores, particularly in some categories. ROC curve analyses showed that the GPT-4 dataset had the highest AUC value 0.80. The study revealed that using synthetic data expands the potential of AI applications in clinical psychology and offers a way to develop effective models while preserving patient privacy. Future research should test synthetic data with different models and compare it with real clinical data.

 Cognitive distortions are thought errors that lead individuals to perceive reality in a misleading way and are strongly associated with psychopathologies. Therefore, accurately identifying and classifying distortions can enhance the effectiveness of cognitive-behavioral therapy CBT. This study investigates the effectiveness of deep learning and NLP techniques for the automatic detection of cognitive distortions. The RoBERTa model was trained using English synthetic data generated by GPT-4 2000 examples and the dataset from Shreevastava and Foltz 1590 cognitive distortion examples, 933 non-distortion examples. Three scenarios were tested: the original dataset, the synthetic dataset, and their combination. The results showed that synthetic data is a strong resource. Accuracy rates were 60.67 original, 94.51 synthetic, and 77.18 combined. The GPT-4-based dataset provided almost perfect F1 scores, particularly in some categories. ROC curve analyses showed that the GPT-4 dataset had the highest AUC value 0.80. The study revealed that using synthetic data expands the potential of AI applications in clinical psychology and offers a way to develop effective models while preserving patient privacy. Future research should test synthetic data with different models and compare it with real clinical data.

 Cognitive distortions are thought errors that lead individuals to perceive reality in a misleading way and are strongly associated with psychopathologies. Therefore, accurately identifying and classifying distortions can enhance the effectiveness of cognitive-behavioral therapy CBT. This study investigates the effectiveness of deep learning and NLP techniques for the automatic detection of cognitive distortions. The RoBERTa model was trained using English synthetic data generated by GPT-4 2000 examples and the dataset from Shreevastava and Foltz 1590 cognitive distortion examples, 933 non-distortion examples. Three scenarios were tested: the original dataset, the synthetic dataset, and their combination. The results showed that synthetic data is a strong resource. Accuracy rates were 60.67 original, 94.51 synthetic, and 77.18 combined. The GPT-4-based dataset provided almost perfect F1 scores, particularly in some categories. ROC curve analyses showed that the GPT-4 dataset had the highest AUC value 0.80. The study revealed that using synthetic data expands the potential of AI applications in clinical psychology and offers a way to develop effective models while preserving patient privacy. Future research should test synthetic data with different models and compare it with real clinical data.

 Bilişsel çarpıtmalar, bireylerin gerçekliği yanıltıcı bir şekilde algılamalarına neden olan düşünce hatalarıdır ve psikopatolojilerle güçlü bir ilişkisi vardır. Bu nedenle, çarpıtmaların doğru bir şekilde belirlenmesi ve sınıflandırılması, bilişsel davranışçı terapinin CBT etkinliğini artırabilir. Bu çalışma, bilişsel çarpıtmaların otomatik tespiti için derin öğrenme ve NLP tekniklerinin etkinliğini incelemektedir. GPT-4 ile üretilen İngilizce sentetik veriler 2000 örnek ve Shreevastava ve Foltzun veri seti 1590 bilişsel çarpıtma, 933 çarpıtma içermeyen örnek kullanılarak RoBERTa modeli eğitilmiştir. Üç senaryo test edilmiştir: orijinal veri seti, sentetik veri seti ve bunların kombinasyonu. Sonuçlar, sentetik verilerin güçlü bir kaynak olduğunu göstermiştir. Doğruluk oranları sırasıyla 60,67 orijinal, 94,51 sentetik ve 77,18 kombine olarak elde edilmiştir. GPT-4 tabanlı veri seti, özellikle bazı kategorilerde neredeyse mükemmel F1 skorları sağlamıştır. ROC eğrisi analizleri, GPT-4 veri setinin en yüksek AUC değerine 0,80 sahip olduğunu göstermiştir. Çalışma, sentetik veri kullanımının klinik psikolojide yapay zeka uygulamalarının potansiyelini genişlettiğini ve hasta gizliliğini korurken etkili modeller geliştirmenin bir yolunu sunduğunu ortaya koymuştur. Gelecekteki araştırmalar için, sentetik verilerin farklı modellerle test edilmesi ve gerçek klinik verilerle karşılaştırılması önerilmektedir.

Article Details

Creating a Clinical Psychology Dataset with Synthetic Data: Automatic Detection of Cognitive Distortions Classified with NLP

Authors:

Hakkı Halil Babacan | Ramazan Oğuz | Yahya Kemal Beyitoğlu

Publisher:

Fırat University

Main Topics:

Doğal Dil İşleme

Journal:

Fırat Üniversitesi Mühendislik Bilimleri Dergisi Vol.37 Issue 1, pp.83-92

Document Type:

Article

Publication Date:

3/2025

Document Language:

English

Document ID (Org No.):

2001334856

pISSN:

1308-9072

Cognitive distortion

; machine learning

natural language processing

GPT-4

depressionBilişsel çarpıtma

makine öğrenimi

doğal dil işleme

GPT-4

depresyon

Issues

Article Details

Authors: Hakkı Halil Babacan | Ramazan Oğuz | Yahya Kemal Beyitoğlu

Publisher: Fırat University

Main Topics: Doğal Dil İşleme

Journal: Fırat Üniversitesi Mühendislik Bilimleri Dergisi Vol.37 Issue 1, pp.83-92

Document Type: Article

Publication Date: 3/2025

Document Language: English

Document ID (Org No.): 2001334856

pISSN: 1308-9072

Issues

Authors:

Hakkı Halil Babacan | Ramazan Oğuz | Yahya Kemal Beyitoğlu

Publisher:

Fırat University

Main Topics:

Doğal Dil İşleme

Journal:

Fırat Üniversitesi Mühendislik Bilimleri Dergisi Vol.37 Issue 1, pp.83-92

Document Type:

Article

Publication Date:

3/2025

Document Language:

English

Document ID (Org No.):

2001334856

pISSN:

1308-9072