Performance Comparison of Naïve Bayes and SVM Algorithms in Sentiment Analysis on JKN Application Data

Authors

  • Meyti Eka Apriyani Politeknik Negeri Malang, Indonesia
  • Amiruddin Fikri Nur Politeknik Negeri Malang, Indonesia
  • Ely Setyo Astuti Politeknik Negeri Malang, Indonesia

DOI:

https://doi.org/10.30983/knowbase.v4i2.8758

Keywords:

Sentiment Analysis, Naïve Bayes, SVM

Abstract

In 2022, 67.88% of Indonesia's population owned mobile devices. BPJS Kesehatan responded to this trend by launching the Mobile JKN application to provide modern, accessible healthcare services. To drive continuous innovation, BPJS Kesehatan needs insights into user feedback regarding the Mobile JKN application. Given the large volume of reviews, sentiment analysis is employed to classify reviews into positive or negative categories. This study compares the performance of Naïve Bayes and SVM (Support Vector Machine) algorithms in sentiment classification using a dataset from the Mobile JKN application. The dataset consists of 200 reviews labeled by two different raters, yielding 110 positive and 90 negative reviews for the first set and 114 positive and 86 negative reviews for the second set. Testing was conducted using three data split scenarios for training and testing: 70:30, 80:20, and 90:10. Model performance was evaluated using a confusion matrix, with metrics including accuracy, precision, recall, and F1-score. The results show that the Naïve Bayes algorithm achieved its best performance with a 90:10 data split, yielding an accuracy of 85%, precision of 77%, recall of 100%, and F1-score of 87%. Conversely, the SVM algorithm performed best with an 80:20 data split, achieving 93% accuracy, 100% precision, 84% recall, and an F1-score of 91% for the first rater's dataset. For the second rater's dataset, SVM reached optimal performance with a 90:10 data split, yielding 90% accuracy, 100% precision, 80% recall, and an F1-score of 89%. Overall, the comparison highlights that SVM outperforms Naïve Bayes in terms of accuracy and precision, making it more effective for predicting positive sentiment in Mobile JKN application reviews.

References

K. Sutarsih, Tri;Maharani, Statistik Telekomunikasi Indonesia. Jakarta: Badan Pusat Statistik, 2023.

Solechan, “Solechan_Badan Penyelenggara Jaminan Sosial (BPJS) Kesehatan,” Adm. Law Gov. J., vol. 2, no. 4, pp. 686–696, 2019, [Online]. Available: https://doi.org/10.14710/alj.v2i4.686-696.

A. N. Romero, Sri Ratna Suminar, and A. H. Zakiran, “Pemenuhan Hak Pasien BPJS dalam Mendapatkan Pelayanan Antidiskriminasi Dihubungkan dengan UU Rumah Sakit,” J. Ris. Ilmu Huk., pp. 31–36, 2023, doi: 10.29313/jrih.v3i1.2121.

A. Wulanadary, S. Sudarman, and I. Ikhsan, “Inovasi Bpjs Kesehatan Dalam Pemberian Layanan Kepada Masyarakat : Aplikasi Mobile Jkn,” J. Public Policy, vol. 5, no. 2, p. 98, 2019, doi: 10.35308/jpp.v5i2.1119.

R. Br Sagala and V. Hajad, “Inovasi Pelayanan Kesehatan Mobile JKN Di Kantor BPJS Kota Subulussalam,” J. Soc. Polit. Gov., vol. 4, no. 1, pp. 14–23, 2022, doi: 10.24076/jspg.2022v4i1.775.

N. Khotimah, “Pengaruh Kualitas Sistem, Kualitas Layanan, dan Kualitas Informasi Pada Aplikasi Mobile JKN Terhadap Kepuasan Peserta BPJS Kesehatan Di Wilayah JABODETABEK,” J. Akunt. dan Manaj. Bisnis, vol. 2, no. 2, pp. 69–76, 2022.

J. J. A. Limbong, I. Sembiring, and K. D. Hartomo, “Analisis Klasifikasi Sentimen Ulasan pada E-Commerce Shopee Berbasis Word Cloud dengan Metode Naive Bayes dan K-Nearest Neighbor,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 2, p. 347, 2022, doi: 10.25126/jtiik.2022924960.

A. R. Maulana, Y. T. Mursityo, and S. H. Wijoyo, “Analisis Sentimen Kebijakan Penerapan Kurikulum Merdeka Sekolah Dasar dan Sekolah Menengah pada Media Sosial Twitter dengan menggunakan Metode Word Embedding dan Long Short-Term Memory Networks (LSTM),” J-PTIIK, vol. 7, no. 17, 2023, [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/12558.

A. N. Nurkalyisah, A. Triayudi, and I. D. Sholihati, “Analisis Sentimen pada Twitter Berbahasa Indonesia Terhadap Penurunan Performa Layanan Indihome dan Telkomsel,” J. Sist. dan Teknol. Inf., vol. 10, no. 4, p. 387, 2022, doi: 10.26418/justin.v10i4.50858.

B. Liu, Sentiment Analysis Mining Opinions, Sentiments, and Emotions, Second. Chicago: University of Illinois, 2020.

Raksaka Indra Alhaqq, I Made Kurniawan Putra, and Yova Ruldeviyani, “Analisis Sentimen terhadap Penggunaan Aplikasi MySAPK BKN di Google Play Store,” J. Nas. Tek. Elektro dan Teknol. Inf., vol. 11, no. 2, pp. 105–113, 2022, doi: 10.22146/jnteti.v11i2.3528.

B. M. Akbar, A. T. Akbar, and R. Husaini, “Analysis of Sentiments and Emotions about Sinovac Vaccine Using Naive Bayes,” Telematika, vol. 19, no. 2, p. 185, 2022, doi: 10.31315/telematika.v19i2.7601.

F. R. Mahardika et al., “Rekomendasi Pengembangan Fasilitas Wisata Tugu Pahlawan Surabaya Melalui Visualisasi Dashboard Hasil Klasifikasi Analisis Sentimen Ulasan Pengunjung,” J. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 2, pp. 363–372, 2022, doi: 10.25126/jtiik.202295655.

Y. Ansori and K. F. H. Holle, “Perbandingan Metode Machine Learning dalam Analisis Sentimen Twitter,” J. Sist. dan Teknol. Inf., vol. 10, no. 4, p. 429, 2022, doi: 10.26418/justin.v10i4.51784.

R. Tineges, A. Triayudi, and I. D. Sholihati, “Analisis Sentimen Terhadap Layanan Indihome Berdasarkan Twitter Dengan Metode Klasifikasi Support Vector Machine (SVM),” J. Media Inform. Budidarma, vol. 4, no. 3, p. 650, 2020, doi: 10.30865/mib.v4i3.2181.

L. M. Putri and Y. Nataliani, “Analisis Sentimen Masyarakat Terhadap Penggunaan Vaksin Covid-19 Di Indonesia Menggunakan Metode Naïve Bayes,” Indones. J. Intell. Data …, vol. 01, no. 01, pp. 1–14, 2023, [Online]. Available: https://ejournal.unsrat.ac.id/index.php/IJIDS/article/view/48609.

V. K. S. Que, A. Iriani, and H. D. Purnomo, “Analisis Sentimen Transportasi Online Menggunakan Support Vector Machine Berbasis Particle Swarm Optimization,” J. Nas. Tek. Elektro dan Teknol. Inf., vol. 9, no. 2, pp. 162–170, 2020, doi: 10.22146/jnteti.v9i2.102.

I. Firmansyah and B. H. Hayadi, “Analisis Sentimen Citayam Fashion Week menggunakan Support Vector Machine,” J. Sist. dan Teknol. Inf., vol. 10, no. 4, p. 513, 2022, doi: 10.26418/justin.v10i4.56665.

D. D. Nada, R. M. Atok, and A. P. Data, “Perbandingan Analisis Sentimen Mengenai BPJS pada Media Sosial Twitter Menggunakan Naïve Bayes Classifier (NBC) dan Support Vector Machine (SVM),” J. SAINS DAN SENI ITS, vol. 11, no. 6, 2022, [Online]. Available: https://t.co/2nUaexGu5i.

M. Birjali, M. Kasri, and A. Beni-Hssane, “A Comprehensive Survey On Sentiment Analysis: Approaches, Challenges And Trends,” Knowledge-Based Syst., vol. 226, 2021, doi: https://doi.org/10.1016/j.knosys.2021.107134.

Friska Aditia Indriyani, Ahmad Fauzi, and Sutan Faisal, “Analisis Sentimen Aplikasi Tiktok Menggunakan Algoritma Naïve Bayes dan Support Vector Machine,” TEKNOSAINS J. Sains, Teknol. dan Inform., vol. 10, no. 2, pp. 176–184, 2023, doi: 10.37373/tekno.v10i2.419.

Downloads

Published

2024-12-31

How to Cite

Eka Apriyani, M., Fikri Nur, A., & Setyo Astuti, E. (2024). Performance Comparison of Naïve Bayes and SVM Algorithms in Sentiment Analysis on JKN Application Data. Knowbase : International Journal of Knowledge in Database, 4(2), 180–188. https://doi.org/10.30983/knowbase.v4i2.8758

Issue

Vol. 4 No. 2 (2024): December 2024

Section

Articles

Citation Check

License

Copyright (c) 2025 Meyti Eka Apriyani, Amiruddin Fikri Nur, Ely Setyo Astuti

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).