THE COMPARISON BETWEEN LECTURER AND AI-POWERED PLATFORM' FEEDBACK IN ASSESSING ENGLISH PRONUNCIATION PDF Free Download
1 / 113/113
100%
THE COMPARISON BETWEEN LECTURER AND AI-
POWERED PLATFORM’ FEEDBACK IN ASSESSING
ENGLISH PRONUNCIATION
(A Comparative Analysis Research at 2nd Semester Students Department of English Education
UIN Syarif Hidayatullah Jakarta Academic Year 2023/2024)
A Skripsi
Presented to the Faculty of Tarbiyah and Teacher Training in Partial Fulfillment of the
Requirements for the Degree of S1 (S.Pd.) in Department of English Education
Supervisor:
Ummi Kultsum, M.Pd., Ph.D.
By:
Alfina Nuryah
11200140000046
DEPARTMENT OF ENGLISH EDUCATION
FACULTY OF TARBIYAH AND TEACHER TRAINING
UIN SYARIF HIDAYATULLAH JAKARTA
2024
i
APPROVAL SHEET
ii
ENDORSEMENT SHEET
iii
STATEMENT OF AUTHENTICITY
iv
ABSTRACT
Alfina Nuryah. NIM: 11200140000046. “The Comparison Between Lecturer and AI-
Powered Platform’ Feedback in Assessing English Pronunciation” (A Comparative
Analysis Research at 2nd semester Students Department of English Education UIN Syarif
Hidayatullah Jakarta). A skripsi of Department of English Education, the Faculty of
Tarbiyah and Teacher Training, Syarif Hidayatullah State Islamic University Jakarta,
2024.
Advisor: Ummi Kultsum, M.Pd., Ph.D.
Since feedback in pronunciation learning is a crucial aspect and AI widely pop up with
sophisticated technology in ELT, this study explores two feedback, AI automatic
feedback from BoldVoice App and lecturers’ corrective feedback. This study aims to
identify the differences and similarities of the feedback and seek whether AI can replace
teacher or be an integrated tool to provide feedback in pronunciation assessment. This
research was carried out through qualitative method with text comparative analysis. The
instruments used in this research were documentation and semi-structured interview. The
research was conducted at UIN Syarif Hidayatullah Jakarta. As this research used random
sampling, six 2nd semester students of English Education Department were randomly
choosen to be the participants of the research. This study shows that both feedback only
shares one similarity goal to help English learners improve their pronunciation, but they
have few differences such as while AI can provide immediate and objective feedback,
lecturer provide personal-conversation subjective feedback with motivational value.
Furthermore, based on in-depth interview, the research found that AI cannot replace
human (Re: teachers/lecturer) as a feedback provider but can help them in giving material
and track students’ progress. It is hoped that this study can provide positive views to
pronunciation instructors and learners due to massive AI technology appearing in this
modern era.
Keyword: Pronunciation feedback, AI automatic feedback, Direct corrective feedback
v
ABSTRAK
Alfina Nuryah. NIM: 11200140000046. “The Comparison Between Lecturer and AI-
Powered Platform’ Feedback in Assessing English Pronunciation” (A Comparative
Analysis Research at 2nd semester Students Department of English Education UIN Syarif
Hidayatullah Jakarta). A skripsi of Department of English Education, the Faculty of
Tarbiyah and Teacher Training, Syarif Hidayatullah State Islamic University Jakarta,
2024.
Advisor: Ummi Kultsum, M.Pd., Ph.D.
Karena feedback dalam pembelajaran pronunciation adalah aspek penting dan teknologi
AI semakin berkembang dalam pengajaran bahasa Inggris (ELT), penelitian ini
mengeksplorasi dua jenis feedback, yaitu feedback otomatis dari aplikasi BoldVoice dan
feedback korektif dari dosen. Penelitian ini bertujuan untuk mengidentifikasi perbedaan
dan persamaan dari kedua jenis feedback tersebut serta mencari tahu apakah AI dapat
menggantikan peran dosen atau berfungsi sebagai alat tambahan dalam memberikan
feedback dalam asesmen pronunciation. Penelitian ini dilakukan dengan metode kualitatif
melalui analisis komparatif teks. Instrumen yang digunakan dalam penelitian ini meliputi
dokumentasi, wawancara semi-terstruktur, dan observasi. Penelitian ini dilaksanakan di
UIN Syarif Hidayatullah Jakarta. Dengan menggunakan metode random sampling, enam
mahasiswa semester 2 dari Program Studi Pendidikan Bahasa Inggris dipilih secara acak
sebagai peserta penelitian. Hasil penelitian menunjukkan bahwa kedua jenis feedback ini
hanya memiliki satu kesamaan, yaitu tujuan untuk membantu pembelajar bahasa Inggris
meningkatkan pronunciation mereka, namun terdapat beberapa perbedaan. Misalnya, AI
dapat memberikan feedback yang langsung dan objektif, sementara dosen memberikan
feedback yang bersifat personal dan subjektif dengan nilai motivasional. Lebih lanjut,
berdasarkan iinterview mendalam, penelitian ini menemukan bahwa AI tidak dapat
menggantikan manusia (dalam hal ini, dosen) sebagai pemberi feedback, namun AI dapat
membantu dalam memberikan materi dan melacak kemajuan belajar siswa. Diharapkan
penelitian ini dapat memberikan pandangan positif kepada dosen dan pembelajar
pronunciation mengingat kemajuan teknologi AI yang semakin pesat di era modern ini.
Kata kunci: Umpan balik pronunciation, Umpan balik otomatis AI, Umpan balik
korektif langsung
vi
ACKNOWLEDGEMENT
In the name of Allah, the Most Gracious, the Most Merciful
First and foremost, all praise and gratitude to Allah SWT, the Lord of the worlds,
for His abundant blessings and mercy, which have guided me throughout this journey and
enabled me to complete this thesis. Without His divine guidance and support, this thesis
would not have been possible. I would also like to express my sincerest prayers and
blessings upon the beloved Prophet Muhammad (peace be upon him), whose teachings
inspire and illuminate our path toward knowledge and understanding. May Allah SWT
continue to bestow blessings upon him, his family, and his companions.
This journey would not have been possible without the support, guidance, and
encouragement of several incredible people who have shared in this journey with me. My
heartfelt thanks to my lovely parents, Ayah and Mama, who have always been my pillars
of support. Their prayers, encouragement, and unwavering belief in my abilities have
been my source of strength. Without them, I would never have reached this point. I am
also grateful to my siblings, my little sister Fida and my not so-called little brother
Kamael, who have always been there to cheer me up during challenging times with their
silliness. Moreover, I would like to extend my sincere thanks to Mrs. Ummi Kultsum,
M.Pd., Ph.D., my thesis supervisor, whose guidance, knowledge, and patience have been
invaluable in the completion of this research. I am especially grateful for her willingness
to provide me with insights, suggestions, and corrections which have shaped this research.
Thus, I could learn many things from her since the proposal until this thesis's completion.
Lastly, I offer my regards and blessings to all those who supported me in any respect
during the completion of the thesis. The researcher’s gratitude also expressed goes to:
1. The Rector of State Islamic University Syarif Hidayatullah Jakarta, Prof. Asep
Saepudin Jahar, M.A., Ph.D.
2. The Dean of Faculty of Educational Sciences UIN Syarif Hidayatullah Jakarta, Prof
Siti Nurul Azkiyah, M.Sc., Ph.D.
3. The Chairman of the English Education Department, Ummi Kultsum, M.Pd., Ph.D.
vii
4. The Academic Advisor of Class B in the academic year of 2020/2024, Prof. Dr. Alek,
M.Pd.
5. The Researcher’s Advisor, Ummi Kultsum, M.Pd., Ph.D.
6. The Researcher’s Examiner 1, Dr. Ismalianing Eviyuliwati, M.Hum.
7. The Researcher’s Examiner 2, Viviana Lisma Lestari, M.Pd.
8. All lecturers and staff of the Department of English Education for the knowledge,
motivation, inspiration, and experience during the researcher’s study.
9. All the participant of 2nd semester students of Department of English Education
academic year 2023/2024 who have given their time to be the sample of this skripsi.
10. Her beloved friends, Syifa, Zhara, Suci, Dinda, Fitri, Nadiah, Zulfa, Yuni, and Faza.
Thank you for huge support, motivation, and love for the whole semesters. The
moments we shared together were beautiful.
11. Her best classmates from class B, thank you for every single wonderful experiences
together.
12. The Widjaja family, Pak Jeffry, Bu Henny, Kak Jenni, Kak Fani, and Kak Esa. Thank
you for great support and love.
13. Everyone who supported and contributed to the completion of this skripsi, whose
names cannot be listed individually.
14. Last but not least, the researcher wants to thank herself. I am thanking myself for
being so strong, for always believing myself, for always grateful, for all the hard
work I done in my live, for have made it up to this point, and for always be myself.
Jakarta, October 2024
Sincerely,
Alfina Nuryah
viii
TABLE OF CONTENTS
APPROVAL SHEET ....................................................................................................... i
ENDORSEMENT SHEET ............................................................................................ ii
STATEMENT OF AUTHENTICITY ......................................................................... iii
ABSTRACT ................................................................................................................... iv
ACKNOWLEDGEMENT ............................................................................................ vi
LIST OF TABLES & FIGURES ................................................................................... x
CHAPTER I .................................................................................................................... 1
INTRODUCTION .......................................................................................................... 1
A. Background of the Research ............................................................................. 1
B. Identification of the problem ............................................................................ 3
C. Limitation of the Research ............................................................................... 4
D. Research Question ............................................................................................ 4
E. Objective of the Research ................................................................................. 5
F. Significance of the Research ............................................................................ 5
CHAPTER II .................................................................................................................. 6
LITERATURE REVIEW .............................................................................................. 6
A. The Importance and Challenges on English Pronunciation.............................. 6
B. English Pronunciation Assessment ................................................................... 7
C. The Role of Feedback in the Learning Process .............................................. 10
D. Feedback in English Language Learning ....................................................... 11
E. Comparing Teacher and AI Feedback in Pronunciation ................................ 14
F. Previous Study ................................................................................................ 15
G. Theoretical Framework................................................................................... 17
CHAPTER III ............................................................................................................... 18
RESEARCH METHODOLOGY ................................................................................ 18
A. Place and Time of the Research ..................................................................... 18
B. Research method and Design ......................................................................... 18
ix
C. Population and sample .................................................................................... 19
D. Instruments of the Research ........................................................................... 20
E. Data Collection ............................................................................................... 23
F. Data Analysis .................................................................................................. 24
CHAPTER IV ............................................................................................................... 27
RESEARCH FINDINGS & DISCUSSION ................................................................ 27
A. Research Findings .......................................................................................... 27
1) Result of Compared Feedback ........................................................................ 27
2) Result of In-Depth Interview .......................................................................... 37
B. Discussion ....................................................................................................... 44
CHAPTER 5 .................................................................................................................. 54
CONCLUSION & SUGGESTIONS ........................................................................... 54
A. Conclusion ...................................................................................................... 54
B. Suggestion ...................................................................................................... 55
C. Implication ...................................................................................................... 55
REFERENCES ............................................................................................................. 58
APPENDICES ............................................................................................................... 66
x
LIST OF TABLES & FIGURES
Table 1 Demographic data of participants ...................................................................... 20
Table 2 Blueprint of Semi-structured Interview ............................................................. 22
Figure 1 Overall Score .................................................................................................... 28
Figure 2 Speech Level .................................................................................................... 28
Figure 3 Top Focus Area ................................................................................................ 28
Figure 4 Top Streghts ..................................................................................................... 28
Figure 5 Detailed Result ................................................................................................. 29
Figure 6 Color-coded score ............................................................................................ 29
Table 3 Detailed data score from 6 test takers................................................................ 30
Table 4 Top 3 strengths of Venus ................................................................................... 30
Table 5 Top 5 focus areas of Venus ............................................................................... 30
Table 6 Detailed sentences-score results of Venus ........................................................ 31
Figure 7 Process Flow on how BoldVoice App works .................................................. 31
Table 7 Speech Levels Description ................................................................................ 34
Table 6 Speech Level from 6 test takers......................................................................... 34
Figure 8 BoldVoice App Plan ........................................................................................ 36
Table 7 Traditional vs AI Scores ..................................... Error! Bookmark not defined.
1
CHAPTER I
INTRODUCTION
A. Background of the Research
English for Academic Purposes (EAP) teachers nowadays have a
difficult time figuring out when and how to give pronunciation feedback to
their students (Baker & Burri, 2016). On the other hand, students also often
face challenges in receiving feedback directly from the teacher to measure
their pronunciation. This issue can arise due to various factors, such as large
class sizes, limited individual attention, teachers’ cognition, subjectivity bias,
and time constraints (Couper, 2019; Padilla & Lee, 2019). In addition, the
issue of who should correct them also becomes one of the factors. As teachers
seen it as less threatening, according to Couper (2016) and Roothooft (2014)
(as cited in Couper, 2019), teachers preferring to use peer and self-correction,
or maybe not even bothering with errors that students made-up at all.
However, new ways to overcome this problem have been opened by
technological advances, especially in the field of Artificial Intelligence (AI),
but Bulut (2022), highlights the limited number of studies discussing the
implementation of AI in feedback generation and delivery processes.
AI-powered language learning platforms and applications can provide
automated feedback (Evenddy, 2024; Saraswati et al., 2023) allowing
students to practice and receive instant feedback. AI system use sophisticated
algorithms to analyze speech patterns and compare them with native speaker
models to help students assess their accuracy and adjust accordingly (de la
Vall & Araya, 2023). In addition, based on the search results on Google and
YouTube, it appears that the use of AI in pronunciation classes is becoming
more common, but it is still not widely used. It may happen due to limited
access to technology, lack of awareness from teachers and students, the cost
of using them, reliance on traditional methods, and concern about the
effectiveness of AI-powered feedback.
2
Research issues related to feedback, either teachers’ corrective
feedback or AI-powered feedback is important to be discussed. For instance,
students who face difficulties in improving their pronunciation may feel
frustrated and lose motivation in learning. Also, without specific and direct
feedback, students may not have an accurate understanding of the progress
they have made. These issues can inhibit their progress. Due to its issues, this
topic has certain urgencies’ including address inequalities regarding access to
quality feedback, improving students’ efficiency in pronunciation learning,
encouraging motivation and confidence to improve students’ pronunciation,
and encouraging innovation in education by employing AI technology
especially to improve pronunciation learning. Through leveraging AI
technology, students will be able to receive more personalized and in-depth
feedback, despite limiting factors that may appear in traditional feedback
methods conducted by teachers.
There are several possible negative impacts if these issues are not
addressed, including limitations in developing pronunciation. According to
Saeli (2021), students who get positive perception on quality feedback from
the teacher, they had significantly higher accuracy gains on their
pronunciation, yet it inverse with whom not. It clearly stated that without
quality feedback and opportunities for direct corrective feedback from
teachers, students may have difficulty in developing good pronunciation.
Saito (2021) also argues that corrective feedback (CF) becomes an important
component in the development of L2 pronunciation, especially when the
errors involved hinder successful communication. Furthermore, without
proper feedback, students may lose motivation and confidence in improving
their pronunciation. Low motivation can hinder students' active participation
in learning and reduce their interest in improving their pronunciation.
In this research, the focus will be on the comparison between the
feedback provided by the lecturer and BoldVoice App as an AI tool.
According to Fudholi & Suominen (2018) feedback system with features for
3
pronunciation such as scoring, speech replay, and providing examples was
considered important by language learners. Therefore, the importance of this
research lies in the quality of the feedback, for instance provides insight into
the best way to provide quality feedback in pronunciation assessment either
one of them or might collaborate. Thus, this research may help improve the
quality of language learning and provide more efficient alternative solutions
in providing feedback to students.
As previous studies above, it has shown that feedback is tied to
students’ learning progress. Lack of feedback on students’ pronunciation can
inhibit students’ progress in improving their pronunciation and may lead to
persistent errors going unnoticed. Due to its important, the writer interest to
comparing between both by conduct this study entitled “The Comparison
Between Lecturer and AI-Powered Platform’ Feedback in Assessing English
Pronunciation” to find out the differences and potential itself. Through
comparing these feedback, this present study may see the pros and cons, and
collaborative between AI and human assessor, also to resolve the obstacles
that might appear in traditional evaluation methods or AI-powered systems.
This study will fill in the gap limited number of studies comparing feedback
aspects of Lecturer’s oral corrective feedback and AI in the pronunciation
environment. Meanwhile, the novelty of this study includes the use of
BoldVoice App as an AI application to evaluate students' pronunciation.
B. Identification of the problem
Based on the background of the research described above, the writer
identified several problems including:
1. Due to the challenges that teachers and students might face in the
classroom pronunciation setting, students do not get quality feedback or
even have opportunities for corrective feedback directly from the teacher
on the accuracy of their pronunciation.
4
2. As an alternative tool for teachers and students to measure pronunciation
accuracy and get automated feedback, the use of AI-powered language
assessment is still not widely used in the classroom pronunciation setting.
C. Limitation of the Research
1. Limited to one skill area: This study will only focus on the assessment of
pronunciation in English language learning. Therefore, the generalization
of the results of this study is limited to this context. It does not cover
other aspects of language learning.
2. In terms of participant: This study has limited participant in terms of the
number of students or teachers and location. This may affect the
representativeness of the results and generalization to a wider population.
3. Deficiencies of AI technology: Errors in the speech recognition system
may affect the validity and reliability of the feedback provided by
BoldVoice App as an AI platform.
4. Teachers’ consistency: If the teacher is inconsistent, then comparisons
with the AI platform may not be accurate or meaningful.
5. Lack of objective measures: The use of BoldVoice App that is non-
standardized may not provide an accurate comparison between human
feedback and AI platforms.
D. Research Question
The writer limits the scope of the research by using the following three
research questions as a guide to direct the writer’s focus to relevant and
important topics as outlined:
1. What are the key differences and similarities between Lecturer’s OCF
and BoldVoice App as an AI feedback in assessing students’
pronunciation?
5
2. Can BoldVoice App as an AI feedback replace lecturer place as an
assessor? Or only as a complementary tool?
E. Objective of the Research
The following objectives will guide the author to conduct this research:
1. To identify the key differences and similarities in the feedback provided
by lecturer and BoldVoice App when assessing students' pronunciation.
This includes comparing the criteria, depth, and quality of feedback.
2. To explore whether BoldVoice App can replace lecturer-led assessment,
or whether they can collaborate to provide complementary feedback in
pronunciation assessment.
F. Significance of the Research
English pronunciation is an important aspect of language learning, and
accurate assessment is essential for student progress. By comparing feedback
provided by lecturer and BoldVoice App, the writer expects this study will
shed light on which method provides more precise pronunciation assessment.
In addition, this research expects to gain teacher understanding to integrate
AI-powered platforms into pronunciation classes.
6
CHAPTER II
LITERATURE REVIEW
A. The Importance and Challenges on English Pronunciation
Pronunciation is a crucial aspect of language learning and
communication. In English language itself, pronunciation covers various
elements such as segmental and suprasegmental. Segmental aspects include
elements such as individual sounds and phonemes. Furthermore,
suprasegmental aspect includes such as stress, intonation, and rhythm (Umar
& Aspany, 2024). These aspects play an important role which contributes to
effective verbal communication and helping clear understanding among
English speakers.
The importance of pronunciation cannot be understated, because it is
a vital aspect in the language, significantly impacting how well we
communicate and understand each other’s. Additionally, Clear and accurate
pronunciation is a key to make the interactions among the speakers smoother
and avoiding confusion. As Takidze (2024) highlights, even with proper
grammar and vocabulary, but if our pronunciation is poor, it can still lead us
to misunderstandings. Therefore, developing strong pronunciation is essential
for English learner to achieve success in mastering English language.
Pronunciation is a challenging skill to acquire. It takes time for
students to improve their pronunciation specifically for learners who are not
living in English speaking country. There are several factors that can affect
pronunciation acquisition, including the influence of the first language,
motivation, age, and identity (Wang, 2023). Furthermore, non-native English
speakers encounter specific challenges when learning English pronunciation.
Many research has explored these challenges and their impact on
pronunciation accuracy. For example, some previous studies found out that
the pronunciation problems encountered by EFL learners, such as Arabs who
often mispronounce many sounds of English due to their mother tongue that
7
can affect their pronunciation, especially differences in language features
between English and learners native language (Abdurahimova & Kobilova,
2022; Riswanto, 2022; Syed & Hussein Abdelrady, 2022).
B. English Pronunciation Assessment
English pronunciation assessment is a vital component of language
evaluation that focuses on measuring an individual's ability to accurately
produce and understand English sounds, words, and intonation patterns. It
plays a crucial role in assessing a learner's pronunciation to identify areas for
improvement and providing feedback of their pronunciation abilities
(Harding, 2012). Therefore, accurate pronunciation assessment is crucial for
individuals who looking to improve their English language skills and
communicate effectively.
Furthermore, the assessment of English pronunciation involves
various methods and tools designed to evaluate different aspects of
pronunciation (Pennington & Rogerson-Revell, 2019). These assessments
typically target the areas such as individual sounds, word stress, sentence
rhythm, intonation patterns, and connected speech. English pronunciation
assessment serves several important purposes, such as provides learners with
valuable feedback on their pronunciation, allowing them to identify areas for
improvement and develop effective strategies for enhancing their
pronunciation. It also assists instructors and curriculum designers in tailoring
instructions to meet learners' specific needs and objectives. One significant
study by Fathonah et al. revealed that different types of feedback, such as
recasts and explicit corrections, significantly aided students in recognizing
and rectifying their pronunciation errors (Fathonah et al., 2023).
1) Traditional Pronunciation Assessment Method
Traditional method refers to conventional approach. In this
context, used to assess learners' pronunciation in conventional way and
8
teacher as an assessor. This method typically involves the use of
standardized tests and focuses on specific aspects of pronunciation, such
as individual sounds, stress patterns, and intonation (Babaeian, 2023).
This traditional assessment typically involves exercises such as reading
aloud, conversation prompts, or repetition tasks.
Moreover, in the study by Trofimovich and Isaacs (2016), the
emphasis of this assessment was placed on a holistic evaluation, noting
that pronunciation assessments should go beyond the overall aspects of
language, including sociolinguistic appropriateness and communicative
effectiveness. This suggests that traditional pronunciation assessment is
deeply tied to the human ability to understand context, emotion, and the
nuanced qualities of spoken language.
Another aspect explored in traditional assessment is the focus on
specific linguistic features, such as vowel accuracy. Graham, Buttery,
and Nolan (2016) conducted a study to examine how vowel
characteristics contribute to the intelligibility of L2 speakers. Their
findings indicated that traditional pronunciation assessments are capable
to effectively identifying pronunciation challenges such as specific to
vowel production, an area where automated methods sometimes fall short
due to the complexity of human speech variation. The need for detailed
assessments by skilled assessors is highlighted here. This is suggesting
that human assessors are still best positioned to evaluate this aspect of
pronunciation accurately. This aligns with the study by Derwing and
Munro (2005) which highlights the importance of teacher-led
assessments in identifying specific areas where students struggle with
pronunciation. Their research indicates that teachers can effectively
pinpoint segmental and suprasegmental errors, which can then be
addressed through teachers’ instruction.
9
Furthermore, in a study conducted by Li (2020), the reliance on
teacher feedback was shown to significantly impact students' confidence
levels. The research found that students who received consistent
feedback from their instructors were more likely to engage in
pronunciation practice outside of the classroom. This suggests that
traditional assessment methods can foster a supportive learning
environment that encourages students to take risks in their pronunciation
attempts.
Despite the benefits above, traditional pronunciation assessment
method also has challenges and limitations. A study by Dewa and
Laksman-Huntley (2022) points out that subjective evaluations by
teachers can lead to inconsistencies in assessment outcomes. The authors
argue that while teachers' auditory skills are valuable, they can also
introduce biases that may affect the reliability of pronunciation
assessments. This highlights the need for ongoing professional
development for instructors to ensure fair and accurate evaluations.
Babaeian (2023) also stated that human assessor (e.g. teacher and
examiner) might have some issues such as inconsistencies and biases.
2) AI-Powered Pronunciation Assessment Method
AI-powered pronunciation assessment methods leverage the
capabilities of artificial intelligence and machine learning to evaluate
learners' pronunciation in a more sophisticated and dynamic manner.
These methods utilize advanced technology to provide detailed feedback,
personalized assessment, and real-time analysis of learners'
pronunciation. A study conducted by de la Vall & Araya (2023) revealed
that this AI system use sophisticated algorithms to analyze speech
patterns and compare them with native speaker models to help students
assess their accuracy.
10
Furthermore, one key feature of AI-powered pronunciation
assessment is the use of speech recognition technology (Babaeian, 2023).
Learners' spoken responses are captured and analyzed by the AI system,
which compares their pronunciation against a reference model or
database of native or target language pronunciation. The system
identifies and measures various aspects of pronunciation, including
individual sounds, stress patterns, intonation, rhythm, and fluency.
Additionally, it often provides learners with instant feedback on their
pronunciation accuracy. The system can highlight specific areas of
improvement and offer tips or exercises to help learners refine their
pronunciation. This immediate feedback enables learners to make
corrections and adjustments in real-time, facilitating faster progress and
self-directed learning.
Moreover, AI-powered pronunciation assessment methods can
adapt to individual learners' needs and proficiency levels. The system can
personalize the assessment based on learners' prior knowledge, progress,
and specific pronunciation goals. The entire assessment process,
including speech recognition, analysis and scoring, takes place in a non-
biased environment without human intervention, ensuring optimal
validity (Babaeian, 2023; Cámara-Arenas et al., 2023).
C. The Role of Feedback in the Learning Process
Feedback is essential in the learning process because it helps learners
improve their skills through guidance and correction. One interesting, the role
of feedback that research has shown is feedback can influence students'
motivation in learning process (Ani, 2019; Asnawi. & Wariyati., 2021).
Furthermore, in language learning, feedback can help to enhance
students’ communication skills and language acquisition. According to Herra
& Kulińska (2018), feedback is crucial in the learning process of English as
11
a foreign language because it aids to language acquisition. Also, feedback can
enhance students’ communication skills because the process of feedback
itself requires interaction between students themselves and instructors. For
example, peer-feedback. It allows students to engage in giving and receiving
feedback regarding their performance. This provides opportunities for
interaction and sharing of ideas between learners (Janesarvatan & Asoodar,
2024). Moreover, feedback is instrumental in language achievement as it
provides learners with the necessary information regarding their knowledge
and skills. It helps students understand their strengths and weaknesses, to
guide them toward better performance.
D. Feedback in English Language Learning
Feedback refers to the information, guidance, or response provided to
an individual or a system based on their performance, behavior, or output. It
is a communication process that aims to provide valuable information to help
improve, adjust, or reinforce current actions or outcomes. According to
Ramani (2019), feedback is defined as a regulatory mechanism whereby the
effects of an activity is fed back to modify and improve future actions.
Furthermore, the culture of providing feedback should be improved and
habituated, especially in terms of education. This is align as quoted on her
research that “A strong feedback culture promotes ongoing formal and
informal feedback targeting continuous performance improvement.”
1) Pronunciation Feedback
Pronunciation feedback plays a crucial role in language learning
and teaching, as it helps learners improve their pronunciation and enhance
their speaking skills. Research by Li & Lian (2022) proves “Corrective
feedback is crucial for pronunciation teaching.” Similarly, Lee and Lyster
(2016) demonstrated the same thing.
12
a. Types of Pronunciation Feedback
There are several types of feedback that can be used to help
students improve their pronunciation. According to the studies found,
some of the most common types of corrective feedback include 1)
Clarification Request: The teacher asks the student to repeat or clarify
their pronunciation, indicating that the utterance has been unclear, 2)
Elicitation: The teacher uses questions or prompts to encourage the
student to repeat or correct their pronunciation, 3) Repetition: The
teacher repeats the student's pronunciation to correct their errors,
either verbally or non-verbally (e.g., through facial expressions or
gestures), 4) Explicit Correction: The teacher directly corrects the
student's pronunciation by providing the correct pronunciation, 5)
Recast: The teacher rephrases the student's pronunciation to correct
their errors, while still maintaining the original meaning, 6)
Metalinguistic Feedback: The teacher provides feedback on the
student's pronunciation using e.g., explaining the correct
pronunciation rules (Haryanto, 2015; Rahman et al., 2020).
b. Teachers’ feedback on pronunciation assessment
The feedback provided by teachers in pronunciation
assessment plays a crucial role in guiding learners towards improved
pronunciation. Research suggests that teachers may need training to
properly assess learners' pronunciation and identify critical features to
give feedback (Phuong, 2022). One common pronunciation feedback
is corrective feedback. Teachers’ corrective feedback on
pronunciation is believed to be important in helping students improve
their intelligibility when speaking English, additionally, students have
perceived that teachers' feedback contributes to their mastery of
grammar and pronunciation in speaking (Haryanto, 2015).
13
c. AI Generated Feedback
AI-powered platforms have revolutionized the process of
giving pronunciation feedback by providing automated and accurate
assessments of learners' pronunciation. These platforms utilize
artificial intelligence technologies to analyze and evaluate learners'
pronunciation, offering detailed feedback and suggestions for
improvement.
One of the key advantages of AI-powered platforms is their
ability to provide objective and consistent feedback. By leveraging
advanced algorithms, these platforms can analyze various aspects of
pronunciation (Saraswati et al., 2023), such as phonemes, stress,
intonation, and rhythm, with high precision. This ensures that learners
receive consistent feedback regardless of the time or location of their
practice sessions.
Furthermore, AI-powered platforms can offer real-time
feedback (de la Vall & Araya, 2023), allowing learners to receive
instant guidance on their pronunciation. This immediate feedback
enables learners to make corrections and adjustments while practicing.
It can lead to faster progress and more efficient language learning.
These platforms often incorporate interactive features that engage
learners in meaningful practice. Learners can listen to native speaker
models to help them compare their own pronunciation to the target
and receive feedback on specific areas that need improvement. Some
platforms even provide visual representations of pronunciation, such
as spectrograms or waveforms, to help learners visualize and analyze
their pronunciation patterns.
14
E. Comparing Teacher and AI Feedback in Pronunciation
Teacher and AI are significantly different in terms of giving
pronunciation feedback. Teachers bring unique human attributes such as
empathy, creativity, and moral judgment to the education process, while AI
can enhance the learning experience and provide support, the role of the
teacher remains a critical component of the education system. A synergy
where both teachers and AI bring their unique strengths can create more
efficient, tailored, and inclusive learning experiences.
The use of AI in providing feedback in education has been a topic of
debate. The integration of teacher feedback and artificial intelligence (AI)
feedback in pronunciation instruction represents a promising approach to
enhance language learning outcomes. Teacher feedback is inherently
adaptive, capable of addressing specific learner needs, and supportive of
emotional aspects of learning, as seen in studies such as Martin et al. (2022).
Their research shows that feedback which includes emotional support, such
as empathy and encouragement, can foster a positive learning environment.
This helps students feel motivated and comfortable with the lecturer. Martin
et al. emphasized feedback that is emotionally supportive, clear, and engaging
can help reduce student anxiety and make teacher-student relationship closer.
On the other hand, AI provides immediate, consistent, and scalable
feedback (Saraswati et al., 2023). McCrocklin (2019) explored student
experiences with ASR-based (Automatic Speech Recognition) dictation
practice in a hybrid pronunciation workshop. Students appreciated the
convenience of ASR for immediate practice, although they also faced
challenges, such as recognition errors. These findings highlight how AI can
efficiently provide technical correction while the teacher addresses emotional
support and individualized feedback. Thus, while AI can provide benefits
such as personalized feedback and support, it is unlikely to completely replace
human teachers. AI lacks the emotional intelligence, creativity, and
15
adaptability that human teachers bring to the classroom. Instead, AI should
be seen as a tool to support, rather than replace, the work of human educators.
F. Previous Study
In this research, the researcher uses two previous studies as a
consideration in doing this research. First, It is AI meets ESL: Assessing the
Effectiveness of AI-Enabled Speaking Evaluations in English Acquisition
compared to Traditional Techniques by John Edward Padilla and Kang-Hee
Lee (2019). The paper examines the integration of artificial intelligence (AI)
in English as a Second Language (ESL) education, focusing on the
comparison between AI-driven speaking assessments and traditional
evaluation methods. This research explores how AI tools work in assessment.
It emphasizes the limitations of traditional evaluation approaches, such as
subjective bias and time constraints, and the promise AI holds to provide
consistent and unbiased feedback. AI-powered tools, particularly those
leveraging Natural Language Processing (NLP), are seen as potential game-
changers in delivering precise pronunciation and fluency assessments.
The study compares outcomes from traditional assessments with those
obtained through AI tools. Results show that while traditional methods
provide in-depth and context-specific insights, AI assessments offer more
consistent and immediate evaluation. However, the research also notes
differences between the two systems. For instance, individual students may
receive different scores under each method. It is indicating the need for
further refinement of AI tools to align more closely with human evaluators.
The paper suggests a hybrid model where AI assessments complement
traditional methods, combining the strengths of traditional and AI methods to
provide a more balanced and effective evaluation.
Second, it is the research by Sadji Evenddy (2024) entitled
Investigating AI's Automated Feedback in English Language Learning. The
16
purpose of the research is to assess the effectiveness of AI-driven feedback in
improving grammar, vocabulary, and pronunciation, while also evaluating
learner engagement and retention. Additionally, the study identifies
challenges and gaps in the use of AI feedback and suggests future directions
for both technological and pedagogical improvements.
They employ a mixed-methods approach to ensure a comprehensive
synthesis of data. It focuses on academic sources published within the last 10
years, including peer-reviewed journals and databases like Google Scholar,
IEEE Xplore, and JSTOR. The analysis integrates qualitative methods such
as thematic synthesis and quantitative measures to evaluate the effectiveness
of AI tools. The result highlights that AI feedback often surpasses traditional
methods in terms of speed, availability, and personalization. However, it also
identifies significant challenges, including issues with accuracy, dependency
on extensive datasets, and institutional resistance, which could inhibit the
broader adoption of AI in educational settings. Additionally, the paper
explores potential technological advancements, underscores the importance
of blending human and AI feedback for optimal outcomes, and identifies
research gaps that suggest areas for further investigation.
17
G. Theoretical Framework
English
Pronunciation
The Importance of
Pronunciation
The Challenges in
Learning
Pronunciation
Pronunciation
Assessment
Get Feedback for
Improvement
Types of
Pronunciation
Feedback
AI Generated
Feedback
Lecturer Oral
Corrective
Feedback
18
CHAPTER III
RESEARCH METHODOLOGY
A. Place and Time of the Research
This present study was held on March – June 2024. The research
carried out in English Education Department, Faculty of Tarbiya and Teacher
Training, State Islamic University Syarif Hidayatullah Jakarta.
The researcher chose the department because: 1) It is a department of
English education, as it is known that English pronunciation is taught and as
a mandatory for students there to take and achieve Sarjana, 2) The researcher
has been taken a preliminary interview to few of the students at English
Education Department, Faculty of Tarbiya and Teacher Training, State
Islamic University Syarif Hidayatullah Jakarta about pronunciation feedback
in pronunciation classroom setting, 3) The department’s vision and mission
are in line with the research theme, which is emphasizing on English language
proficiency and English teacher training, where pronunciation is part of the
crucial skills that must be improved, also 4) The department is organizing
quality of higher education in the field of English education with ICT support.
B. Research method and Design
A qualitative method is used to address this research as well as
Creswell (2018) stated on his book “Qualitative is best suited to address a
research problem in which you do not know the variables and need to
explore.” In addition, regarding qualitative research, it begins in the field,
which is based on the natural environment to seek understand and explore
(Nassaji, 2020) rather than to explain and manipulate variables. Usually, the
data and information obtained from the field is presented in a descriptive
analysis and not numerically, as it emphasizes the processes occurring in the
field.
19
In this present study, the type of research is comparative analysis.
According to Coccia (2018), comparative study is investigation that analyze
and evaluate phenomena and/or facts among different areas, subjects, and/or
objects to detect similarities and/or differences to gain a deeper understanding
of them. Comparative studies offer a valuable tool for researchers to explore
and compare different aspects of language learning, teaching, and other
phenomena across various contexts, providing insights into the similarities
and differences between them.
This comparative study is to show and compare pronunciation
feedback assessment between traditional-led and AI-led, including the
criteria, model, the pros and cons, the experiences and perspective of the
students.
C. Population and sample
The researcher used random sampling. She selected 6 random students
of 2nd semester at English Education Department, UIN Syarif Hidayatullah
Jakarta as the participant, and to complete the data the researcher interviewed
each of them. Furthermore, the researcher asked their pronunciation lecturer
to help to be a human assessor who gives pronunciation feedback. The
researcher chose them as the participant because they are completely linier to
the data that the researcher wanted to explore as they have Pronunciation
class. The researcher used 6 students because they divided into 3 classes and
2 students representing each class.
20
Table 1 Demographic data of participants
Participants
Gender
Pseudonym
P1
Female
Venus
P2
Female
Earth
P3
Female
Mars
P4
Male
Jupiter
P5
Male
Saturn
P6
Male
Neptune
D. Instruments of the Research
An instrument is a tool to measure, observe, or document data that
contains specific questions and possible answers that the researcher might
create and develop before the study. To gather the data, the researcher used
an instrument which was commonly called a data instrument. The key
instrument of this present study is the researcher itself due to her being
directly involved in the study as an observer and interviewer.
Then, to address RQ1, the lecturer provided corrective feedback to
learners based on the pronunciation assessment rubric that she adopted from
Linguaskill Speaking Phonological Descriptors for Read-aloud and
combined with Pronunciation Assessment: Read Aloud rubric from Khon
Kaen University. She used their expertise to identify strengths and
weaknesses in learners' pronunciation and offer targeted suggestions for
improvement. The feedback is provided through written score and directly
spoken qualitative feedback. Then, on the other days, the researcher asked 2
participants from each class to use AI application for pronunciation
assessment, it is BoldVoice App. This app analyzed learners' pronunciation
based on the assessment rubric itself and provided automated feedback on
21
specific areas that need improvement. The feedback can be in the form of
written comments, audio samples, or visual representations. The researcher
chose this AI app because there is no similar research that use BoldVoice App
as an AI application in term of pronunciation assessment and pronunciation
feedback. Additionally, BoldVoice App is kindly new rather than others app
such as Elsa Speak, and BoldVoice App includes reading aloud assessment
similar to the assessment that was carried out in the pronunciation classrooms
at Department of English Education, UIN Syarif Hidayatullah Jakarta.
On the other hand, semi-structured interviews were conducted to
address RQ. The interview guidelines was adapted from Umarella (2023) on
her research article that was conducted on English Department Students
enrolling in Speaking 4 class at English Education Study Program, Pattimura
University (see Table 2) regarding students' perspective of each feedback
method. In addition, the documentation was documented by voice recorder
and camera to record their answers and documented their real time activities
so that it helped to gain the information. Last, the recorded data analyzed to
depict their perception.
22
Table 2 Blueprint of Semi-structured Interview
Indicator
Question
The Role of Feedback in
Language Learning
Could you describe your understanding of the role and
importance of feedback in the learning process?
The Importance of Feedback
on Pronunciation
How important is feedback for learners’ pronunciation
ability?
The Impact of Feedback on
Pronunciation
How do you believe feedback impacts learners’
pronunciation in language learning?
Whether feedback can engage students to improve their
pronunciation ability and does the lecturer interrupt the
student during student’s pronunciation performance to
correct the pronunciation?
Types of Pronunciation
Feedback
What kind of feedback that can help learners improved the
English pronunciation? Please give the reason!
Have you ever got feedback from lecturer on your
pronunciation in classroom setting? If you have, can you
share your personal experiences regarding the feedback?
What kind of feedback from the lecturer that you think will
help your pronunciation? Do you prefer it in formal or
informal way? Written or spoken feedback?
AI on Pronunciation Practice
Has your lecturer ever used AI in the pronunciation
classroom? If it is no, have you use AI to practice or assess
your English pronunciation?
Comparing AI Feedback and
Lecturers' Direct Corrective
Feedback
In your opinion, what are the strengths and limitations of
feedback provided by artificial intelligence (AI) compared
to feedback given by a lecturer? How do you prefer to
receive feedback, and why?
AI Replacement Role
Do you agree that AI can replace lecturer on giving
feedback in EFL setting?
BOLDVOICE APP
LECTURER OCF
STRUCTURE
APPROACH
ACCESSIBILITY
COST
STRENGTH
LIMITATION
Table 2.1 Aspects of Comparison
23
E. Data Collection
The researcher used some procedures to conduct this research. This step was
taken to make research more manageable. Because the instrument is related, the
entire approach was designed to collect data from pronunciation assessment to get
feedback’ documents that researcher used for document analysis and semi structure
in-depth interview.
The researcher collected the data from two sources: 1) Feedback’
documents from lecturer as human assessor who given the direct corrective
feedback and a BoldVoice app as an AI which given the automatic feedback; and
2) Students’ interview. To begin, a sample of students were selected randomly, and
pronunciation assessments were conducted the next week. The researcher provided
10 sentences which were adopted from BoldVoice App, then each of them were
asked to read aloud each sentence in front of the lecturer and then the lecturer gives
them score and direct feedback regarding their pronunciation performance. On the
next day, the 2 students from each class were asked to do AI pronunciation
assessment to get automatic feedback from AI. After that, the writer conducted
interviews to get their perspective towards the feedback from each method.
To collect the data, the following techniques were conducted:
1) Documentation
In this research, to get a brief description and understanding about the
research focus, a number of documents such as written feedback, spoken
feedback, and assessment process were documented. These were analyzed to
deepen and specify the research results.
1. Traditional Feedback: Lecturer graded and provided corrective feedback
based on students’ pronunciation assessment. The feedback was
documented, it included written score and direct spoken feedback for
improvement.
2. AI-powered Automated Feedback: In the same way, BoldVoice App
provided automated feedback on learners' pronunciation based on their
assessment through the app. The feedback generated by the AI app was
documented and collected for analysis.
24
2) Interview
The interview for this research used semi-structured formats. The
interview was conducted after each student got their pronunciation feedback
from lecturer as traditional method and AI driven method to see students’
perspective towards each feedback method. This qualitative data provided
insights into students' experiences, preferences, perceived benefits, and pros
and cons of each feedback method.
F. Data Analysis
According to Ian Dey (1995), data analysis is the process of breaking down
the data into its constituent components to reveal its elements, characteristics, and
structure. Then, combine them to gain a new understanding. Following the
collection of data through feedback given from the pronunciation assessments and
interviews, the data was analyzed in order to decide which data will be included
and covered in the findings.
According to Iranifard & Roudsari (2022) “to analysis comparative
research, quantitative or qualitative approach can be used” and in this present study
the researcher applied qualitative approach analysis. This comparison focuses on
textual comparative analysis in which the dataset from automatic feedback and
transcript direct feedback from lecturer were documented and presented into a text
file. Since comparative with qualitative approach analysis applied on this study, the
findings of this research study are presented in a narrative form.
Furthermore, the data analysis technique that the researcher applied to
analyzed transcript interviews is Miles and Huberman (1994) model. It consists of
four steps and none of them require statistical methods. Data analysis by Miles and
Huberman model included the following steps:
1) Data Collection
Data collection has a vital role in research. In the context of data
collection, this involves gathering the necessary information and it was
25
interactive and repetitive to ensure a comprehensive understanding. The writer
collected the data from pronunciation assessments and continued with semi-
structured interviews. It was done to 6 random students who attend
Pronunciation subject.
The assessment was conducted to get the feedback form either direct
feedback from lecturer or automatic feedback generated by AI. Next, semi-
structured interview was conducted to get students’ perception, it can be
positive or negative towards both feedback form. Furthermore, the interview
was in-depth formatted, this was done to find out not only their perceptions,
but also to find out whether the assessment and feedback generated by AI can
completely displacing the position of human assessor or possibly to be
collaborative tools.
2) Data Reduction
Data reduction is a critical process in data analysis that aims to
minimize the data while keeping its essential information. Data reduction
includes various techniques and approaches designed to make large datasets
more manageable, efficient, and easier to analyze. This is in line with the
Miles & Hubermen (1994) statement, data reduction involves the process of
selecting, concentrating, simplifying, abstracting, and transforming the raw
data.
After conducting the interviews, the researcher transcribes the audio
of interviews. Then, simplify the data by coding the transcription form. Some
key themes emerged regarding the feedback in English language learning,
particularly in the area of pronunciation. Also, students’ perception towards
that feedback.
3) Data Display
Data display according to Miles and Huberman (1994) is a step in
qualitative data analysis that aims to present the data in a format such as
tables, diagrams, or matrices that allows researchers to clearly see and
26
interpret the data. In this present study, after the interview data is collected
and reduced, the data is presented in a table format. It makes it easy for the
researcher to identify patterns, themes, or connections that appear from the
data analyzed. This part is a crucial step because data display is not only
displaying the information that appears but also an analytical tool that helps
the researcher draw the conclusion and verify the data.
4) Data Verification
Data verification is the last step from Miles & Huberman model in
qualitative data analysis. Data verification in qualitative research involves
several important steps to ensure the reliability and validity of the data
obtained, including triangulation and rich data.
27
CHAPTER IV
RESEARCH FINDINGS & DISCUSSION
In this chapter, she will discuss the findings from her qualitative research. These
findings are collected from data through in-depth interviews and document reviews. This
chapter provides the findings of AI generated feedback vs traditional direct corrective
feedback and students’ perception towards them.
A. Research Findings
As discussed in the previous chapter, researcher conducted this present study at
UIN Syarif Hidayatullah Jakarta, in the second semester of English Education
Department for six random students. This research was conducted using an in-depth
interview and feedback that were documented from AI generated and lecturer OCF.
The following sub-chapter describes the research results.
1) Result of Compared Feedback
This section describes the feedback provided by an AI-based
pronunciation assessment tool (re: BoldVoice App) and direct oral corrective
feedback from a lecturer through comparative text analysis. The study focuses on
profiles of AI feedback compared to human assessors. The researcher breaks
down the delivery, the content, and what was emphasized by each feedback.
a. Structure of Feedback
The structure of feedback provided by the BoldVoice and the lecturer
are significantly different in terms of structure, detail, and delivery. The
following sub chapters are present descriptively to explore each feedback.
1. BoldVoice App Feedback Structure
The BoldVoice App feedback is highly structured. The grades
cover vowels, consonants and diphthongs. It also organizes the feedback
into strengths and areas that need to be improved. The feedback provides
specific sounds that students got right or wrong. Then, it will be
presented in a visualization report for the performance of each student.
28
Figure 1 Overall Score
Figure 2 Speech Level
Figure 3 Top Focus Area
Figure 4 Top Streghts
It gives an excellent sense of how well they are doing with
pronunciation. Moreover, the feedback presents sentence-specific scores
which allow test takers to see their overall performance on each sentence
rather than only individual sounds of the words.
29
Figure 5 Detailed Result
Furthermore, AI categorizes each sound error into levels using
colors. Green as "Awesome"; yellow as "Nice Job"; orange as "Good
Effort"; and red as "Bad".
Figure 6 Color-coded score
30
The following tables are detailed data scores from all students
(N=6) and an example of the feedback results given by the AI application
(BoldVoice App) to one of the students who did the assessment which
the researcher has moved the visualizations report from the app into a
table form.
Table 3 Detailed data score from 6 test takers
Pseudonym
Overall
Score
Vowels
Score
Consonants
Score
Combined
Sounds
Score
Venus
83%
81%
84%
81%
Earth
83%
0%
83%
83%
Mars
82%
82%
82%
84%
Jupiter
82%
80%
83%
0%
Saturn
88%
87%
88%
90%
Neptune
88%
85%
89%
90%
Table 4 Top 3 strengths of Venus
Sounds
Pronounced correctly in
Score
k
cookies, sink, cooking, thankful, looking, book
87%
l
flowers, yellow, looking, slow, lady, beautiful
86%
n
and, don't, when, no, thin, dinner
86%
Table 5 Top 5 focus areas of Venus
Sounds
Mispronounced in
Score
ə
thankful, purchase, the, beautiful
80%
ɪ
dinner, thin, cooking, things
80%
oʊ
don’t, over, goes
80%
ð
the
81%
u
beautiful, shoes, who's, super, student
81%
31
Table 6 Detailed sentences-score results of Venus
No.
Sentences
Score
1
They were looking for a good book
94%
2
I made three sugar cookies and a great fig cake
82%
3
Who’s the new theater student over there?
66%
4
As the saying goes, face the truth and push through
91%
5
She’s super thankful for the beautiful birthday flowers
86%
6
The days are so long and slow when you’re young
70%
7
Did the thin lady purchase those yellow running shoes?
88%
8
The daughter is cooking another healthy dinner at home
80%
9
No, don’t put those things above the bathroom sink
83%
10
They both had an amazing view of today’s football game
88%
Moreover, the feedback also includes AI Coach, which teaches
the user through video and sound. After taking SpeechScan, this app will
give feedback not only by the score but also by the coach’s sound vs the
test taker’s sound in pronouncing the words and provides a comparison
to similar sounds if the test taker mispronounced it by using a mouth
diagram or full video explanation. Also, the personalization and practice
set are provided after the test taker did a pronunciation assessment on
SpeechScan. It contains challenging words from the words that the test
taker cannot pronounce correctly and gives personalization to enhance
their learning experience by customizing the lessons related to learners’
specific needs. The following figure displays the process flow on how
the BoldVoice App works.
Figure 7 Process Flow on how BoldVoice App works
Sign-up or
Log-in
Initial
Assessment
User
Recording
Speech
Analysis
Feedback
Generation
Personalized
Practice
Progress
Tracking
Continuous
Learning
32
2. Lecturer Feedback Structure
The lecturer's feedback is less structured and qualitative. It is
more conversational and personal. The lecturer gives feedback using
direct comments on student’s pronunciation during live assessment. The
qualitative feedback approach focuses on specific sounds and provides
corrections on the spot. The lecturer also discusses general issues that
affect students' pronunciation. The feedback also offers immediate
suggestions after learners pronounce the words or sentences. For
example, based on corrective feedback that the lecturer has done, the
researcher saw that the lecturer often identifies problematic sounds
across multiple words, such as the voiceless /θ/.
"... overall, it's already good. But the /θ/ sound was still
mixed up. But you pronounced it all became voiced. Like
/boʊθ/, it should be voiceless, right? /ˈhel.θi/ /ˈθæŋk ˌjuː/
should also be voiceless …." – quoted 1 (from live
assessment for Venus)
"I found your accent, the words you pronounced were all
good, especially the /th/ and /d/ sounds, they're good, but
you still need to practice the voiceless ones, like "three"
"thing" "tough" "bathroom" "both", those are voiceless,
right? There still needs to be a little more emphasis." –
quoted 2 (from live assessment for Neptune)
Additionally, the lecturer also notes long and short vowel
distinctions, such as in the diphthong sound /eɪ/ in the words "cake" and
"made."
“…. the word “made” and “cake” those are diphthongs,
right? /eɪ/ so they should be read long.” – quoted 3 (from
live assessment for Mars)
33
Moreover, as lecturer is a human assessor, she can pay attention
to the details of cultural or mother tongue influences, such as the Betawi
accent and Bahasa Indonesia that affecting the pronunciation of the
sound /r/ in words like "over" and sound /ʃ/. Also, the lecturer can pay
attention and feel whether students are nervous or not, because it can
affect their pronunciation performance.
“… you're Betawi, so the /r/ sound in "over" is too strong,
too Betawi dialect, like "overr". It should be "ōvər ðer”.”
- quoted 4 (from live assessment for venus)
“… it wasn't clear enough, the /ʃ/ sound hasn't clear yet
because in Indonesian there's no /ʃ/ sound. We don't say
"syaitan" but "setan" ... So, you said "push" was like /pʊs/,
lacking the /ʃ/ sound, but it’s okay, no problem.” - quoted
5 (from live assessment for Mars)
“Why are you not doing it good in the classroom? ......
Yes, too nervous. No worries. Alright, um, so most of the
words have been pronounced quite well for the 10
sentences, ......” – quoted 6 (from live assessment for
Neptune)
This indicates that feedback from lecturer was more holistic, not
only on individual sounds but also the influence of student’s native
language and the impact of nervousness on pronunciation performance.
b. Feedback Approach
1. AI Feedback Approach
BoldVoice utilizes a quantitative approach to assess
pronunciation. It contains individual sound scores (see Figure 6) and
34
sentence correctness (see Table 6). Furthermore, students' performance
is labelled in levels according to their scores by the AI (see Figure 2).
This approach allows consistent and objective assessment for all
students. The speech level of 6 students who took the test is shown in
table format with detail as follows:
Table 7 Speech Levels Description
Speech levels
90% - 100%
Native
80% - 89%
Advance
70% - 79%
Intermediate
60% - 69%
Beginner
50% - 59%
Novice
Table 6 Speech Level from 6 test takers
Pseudonym
Speech Level
Venus
83%
Earth
83%
Mars
82%
Jupiter
82%
Saturn
88%
Neptune
88%
2. Lecturer Feedback Approach
The feedback provided by the lecturer is more subjective and
qualitative. The lecturer used her expertise and perceptions to observe
student's live performances. This approach allows the lecturer to
consider various aspects, such as nuances that AI might not capture.
Additionally, the feedback given by the lecturer is highly
personal. It often considers the student's specific challenges, including
difficulties with certain sounds, nervousness, the student's culture, and
native language (see quoted 1 to 6). Hence, this personal approach
enables the lecturer to provide more suitable and meaningful guidance to
35
help them overcome their obstacles. The emphasis on this personal
connection also means that each student's feedback may differ. It depends
on their progress and areas of difficulty.
Moreover, at the end of the assessment, the lecturer provides a
final score for the students to score their overall pronunciation.
“…. Oke, thank you very much I give you 80 ….” –
quoted 7 (from live assessment of Jupiter)
This score is not usually the most important, because it only shows their
overall performance. Thus, the feedback explains the strengths and
weaknesses to help them see their strengths, progress, and where they
still need to improve. Furthermore, it is often contextual, connected to
their real life. This approach creates a more supportive learning
environment, focusing on improvement, not scoring numbers.
c. Accessibility
The AI-generated feedback like BoldVoice App offers accessibility.
It is available 24/7 free to access. This platform allows learners to receive
feedback at any time and anywhere. Thus, it is particularly beneficial for
self-paced learners, enabling them to practice and improve their
pronunciation outside of traditional class hours.
Moreover, BoldVoice App can handle a large number of students at
the same time. This scalability feature ensures that learners can receive
immediate responses to their pronunciation and can help the students
eliminate waiting times due to class sizes. Thus, it is potentially accelerating
the learning process. Due to BoldVoice App is a technology supported by
internet-based, so learners can access it from various devices such as
smartphones or tablets as long as it is connected to the internet. This
flexibility allows them to learn on-the-go.
36
On the other hand, the lecturer’s oral corrective feedback differs
from BoldVoice App. The availability is more limited and dependent on the
class schedule. The student must attend a class or session to receive
feedback, which typically happens at specific times based on the lecturer’s
schedule. This type of feedback is less flexible because it requires real-time
and one-on-one interaction, which can be constrained by both the lecturer’s
and student’s availability.
d. The Cost Subscription
Because of BoldVoice App provided its features such as immidiate
feedback, scores, and accessibility, yet it has financial costs for users.
BoldVoice App offers 2 subscription plans, they are:
1. Standard plan costs Rp1,390,000 per year or Rp379,000 monthly. This
plan allowed learners’ access to all lessons without unlimited AI chat
practice
2. Super plan cost at Rp1,839,000 yearly. This plan offers learners access
to all lessons and unlimited AI chat practice
Figure 8 BoldVoice App Plan
These costs can be substantial. It potentially creates a barrier for learners
who do not have much financial resources.
37
Meanwhile, the oral corrective feedback that the participants get
from their lecturer looks free because they only spend money for the whole
semester, including all subjects they attend. Thus, the lecturer's feedback is
coss-less.
2) Result of In-Depth Interview
This part presents the findings from interviews conducted with six
English as a Foreign Language (EFL) learners regarding their perceptions of
feedback in pronunciation learning, with a focus on comparing BoldVoice App
and Lecturer’s feedback. The findings are organized into several key themes that
emerged from the data analysis.
a. Strengths and Limitations of BoldVoice vs. Lecturer’s Feedback
After conducted in depth interview, as in the previous chapter
explained, this focused on feedback by BoldVoice App and lecturer oral
corrective feedback (OCF). Participants were asked strengths and weaknesses
of each feedback. They identified various strengths and limitations as follows:
1. Strengths of BoldVoice App
Venus noted the detail and accessibility of AI feedback: “.... from
AI, for example, the advantage is we can access the application any time
and anywhere. Whenever we need it when we want to learn or have free
time, we can access it. Also, our voice will be recorded. Our given
feedback our assessment in the application will be displayed and
recorded, so it helps us to see our level or our mistakes and strengths in
Pronunciation."
On the other side, Earth highlighted the consistency and speed of
AI in giving feedback: “.... the advantage of AI is its ability to provide
consistent feedback, and immediately ....”
38
Moreover, Mars also mentioned its consistency and objectivity of
AI: "I think AI feedback has strength in consistency and objectivity."
Moreover, Jupiter emphasized the detail and availability of AI
feedback: “…. the explanation is more detailed, sometimes it is very
detailed, like until the smallest points are noticeable. It usually
undetectable by human assessor ….” He also added that “we can do
practice or assessment any time.”
Next, Saturn noted the comprehensive nature of AI feedback:
“The AI is quite complete and detailed in providing feedback.”
2. Limitations of BoldVoice App
Beside the strengths of AI automated feedback which provided
by BoldVoice App, it is also having the limitations.
Venus mentioned the lack potential technical issues from
BoldVoice that supported by technology and internet-based: “.... The
weakness is of course the devices must support the app, especially
network problems. If there is a problem, it will affect the program's
performance.”
Earth pointed out the lack of context understanding and device
dependency: “.... the drawback may be that AI is still lacking in
understanding the context and nuances possessed by a human. So it's
like it's more awkward, and also we have to use a proper device so it's
not laggy, and also the internet, the signal must be good and we must
have enough internet quota.”
Moreover, Mars noted the lack of personalization and empathy:
"But on the other hand, a human lecturer can motivate and encourage
students in a way that a computer program cannot."
39
Jupiter also mentioned potential unreliability and awkward: “….
the negative is sometimes we doubt it, because it is still in the form of AI
maybe, so we doubt the results. We can't trust it, is it true or not?
Because the rubric is unclear, the benchmark is unclear .... It's easy to
get bored. Then it's too lonely, then it's too awkward.”
To add, Saturn pointed out potential technical issues: “.... the
signal. for example, we are in the middle of doing assessment and get
the feedback and we want to know, we want to know our mistakes and
when it is lagging or loading again, the internet isn’t support, we have
to do a retest."
Last, Neptune emphasized the dependency on device quality
because it is crucial. It can disruptive how feedback will be generated.
Also, AI cannot see students’ mother tongue and accent that might affect
their performance: “Technology needs a proper device so that the
results can be optimal. For example, if we learn through BoldVoice, then
if our smartphone is not working, such as the microphone on the phone
is broken or something like that, it will be automatically affected .... and
AI has a programmed accent, for example, only British English.”
3. Strengths of Lecturer’s Feedback
Participants also identified various strengths of lecturer
feedback. Earth mentioned the emotional connection on lecturer
feedback: “.... it can provide a more personalized and in-depth view and
also yesterday the lecturer gave feedback I can feel her motivation, it
cannot be provided by AI.”
Mars pointed out the motivation from the lecturer while giving
the feedback: "…. the lecturer, they can motivate and encourage us."
She also added that “…. the ability of lecturer to personalize feedback”
"And on the other hand, feedback from a lecturer can be more
personalized and empathy and a lecturer can understand students'
40
individual learning style and provide feedback that is easy to learn linier
with students' needs."
Furthermore, Jupiter highlighted the educational background
from the lecturer which makes the feedback given trustworthy: “..., we
know her educational background and where she was graduated, .... it’s
feels more reliable”
Neptune also highlighted lecturer can hear our voice clearly
rather than AI. Moreover, the lecturer also could see students’ mother
tongue and differs accent: “.... our voices can be heard more clearly,
our mouth movements can be directly observed by the lecturer. Also, the
voice of the lecturer when she provided the feedback on how to
pronounce a symbol, we can hear it clearly and the mouth movement,
how she pronounced, will also be clearly visible .... mother tongue, the
difference in accent can be assessed directly by the lecturer ....”
4. Limitations of Lecturer’s Feedback
According to the interview, the researcher also found limitations
of lecturer feedback by the students’ point of view. Venus mentioned
during pronunciation assessment with lecturer sometimes makes her
nervous and it hinder her performance, thus might affect the feedback
she will gets: “.... sometimes with the lecturer there is also a sense of
nervousness because we face-to-face with the lecturer, but actually it
depends on each student. Sometimes the nervousness also affects how
we practice pronunciation in front of the lecturer itself.”
Earth pointed out that when she wants to get the feedback from
lecturer, she feels afraid in front of the lecturer: "…. and I feel afraid too
…." Jupiter highlighted the possibility time constraints: “.... probably
the opposite of AI, so the time is monotonous, the time is not as free as
AI when want to get the feedback, we have to be in class.”
41
Next, Saturn pointed out that different feedback from different
lecturer makes him confused: “.... lacks feedback from lecturers, it may
be less detailed because lecturers are still human, some lecturers also
have differences of opinion so that sometimes makes us confused, for
example from lecturer A 'you are good here' from lecturer B, you are
good in a different section ....”
Last, same as Saturn, Neptune also noted inconsistency feedback
from different lecturer: “…. each lecturer may not be the same in
pronunciation, there may be some differences in each lecturer, so it is
inconsistent ….”
On the other hand, Mars mentioned that she doesn’t see any
weaknesses or limitations from lecturer feedback because she prefers the
feedback from human assessor: “The limitation of lecturer. To be
honest, I have no idea for that because I prefer lecturer feedback.”
b. Human Assessor in Giving the Feedback is Irreplaceable
Most participants disagreed that AI could fully replace human
teachers in giving feedback, yet only one participant agreeing for assessment
purposes. Such as Venus disagreed: “Actually do not fully agree, because in
my own opinion Feedback from a lecturer is also needed because a lecturer
also has a sense of empathy and that makes it different from AI. Although AI
is also made to encourage us better but the direct corrective feedback from
lecturer in my opinion affects the enthusiasm for students or pronunciation
learners in improving their pronunciatio. So, it’s 50:50.”
Furthermore, Earth mentioned she is disagreed with the statement that
AI can replace the lecturer: “For myself, I don't fully agree that AI can
completely replace lecturer in providing feedback in the context of learning
English as a foreign language. Maybe it can be mixed, so we learn with
lecturers and learn with AI too, but if to replace it can't. Even though AI can
42
provide feedback, a useful feedback, the presence of a lecturer is still
important to provide more in-depth and personal guidance....”
Another statement of disagreements from Mars: “…. while AI can be a
helpful tool in providing feedback in an EFL setting, it may not be able to
fully replace the role of our lecturer ….” Jupiter also expressed: “Personally,
I don't agree .... because like I was said before, the importance of interaction
between lecturer and student. So, if it's with a robot (AI), it's awkward, too
quiet, and lonely. Language is an interaction between humans because
humans are social creatures. It’s verry different with AI.”
Next, Neptune also disagreed: “In my opinion, I don't agree. The
problem is, AI is a program, a program that may be designed in such a way
to achieve perfection in an aspect, for example in EFL itself … but of course
there are many shortcomings that are differs from the human side. Us, as a
human, we have emotions, we have empathy, which basically means we have
a heart, and AI doesn't have that. We are humans has the tolerance. It depends
on the condition of each student ….”
On the other side, Saturn agreed that AI for assessment and giving the
feedback but disagreed for teaching: “Okay, if AI can replace lecturer, I
actually agree because times is growing, we will definitely need something
like this for our future development, …. So, I agree .... only focuses on
assessing and giving the feedback, but if to provide theory and various other
things like teaching I prefer lecturer”
c. Integrated the Feedback Approach
The researcher asked further question for students in in-depth
interview to know whether these feedback can be integrated or not and all
students were agreed that a combination of AI and lecturer feedback could be
beneficial in pronunciation learning but most of the students agreed only on
how AI can be supporting tool for alternative learning, not giving the
feedback, and only Saturn choose AI to give automatic feedback.
43
“I think it might be possible, because using the application AI-based
also makes it easier for us to track our pronunciation practice activities not
only in class but also outside of class.” – Venus
“Maybe it can be if for example in giving the material resources, yet
if for giving feedback, I prefer the lecturer. If we want another material
resources, it’s fine from AI.” – Earth
“I think it might be. What is it? Yeah, collaboration between lecturer
and AI. And I think it's going to be more effective like AI can give us the
correct, correct pronunciation, such as maybe the grammar or the
pronunciation of etcetera in some way. And we can also get the motivation
and encouragement from the lecturer, yeah, to be better.” – Mars
“Collaboration is possible, so the lecturer also monitors the feedback
from AI. The lecturer has to knows the result from AI .... still has to check one
by one from each student. Do not depend on AI. So, use AI to help, but not
depend on AI .... taught by the lecturer first, then a few meetings are maybe
we can use AI, but still in the end we should get be assess again by the
lecturer.” – Jupiter
“If this is the option, I prefer to combine anyway. There's a lecturer
to teach us and we will also be able to get feedback from AI as well as from
the lecturer directly, so there will be no feud of opinions that might makes us
confused. It’s also can help lecturer if, for example, the AI has given feedback
directly to us, it can shorten the lecturer’s time in class in giving the feedback
because if there are many students in the class, it can’t be done in one day,
one by one.” – Saturn
“As for the collaborative, actually I completely agree. Because we
already have competent lecturers plus AI assistance that can improve. That
can improve our learning at home.” – Neptune
44
B. Discussion
This chapter addresses the research questions by exploring the differences
and similarities between BoldVoice feedback and lecturer OCF feedback in assessing
students' pronunciation. It also explores students' perceptions of both feedback types
to see whether AI feedback can replace or complement lecturer feedback.
1) BoldVoice App VS Lecturer Oral Corrective Feedback (OCF)
BoldVoice App
Lecturer OCF
Structure
• Highly structured
• Student’s performance
presented in a
visualization report
• Less structured
• Conversational
• Provides corrections on
the spot
• Offers immediate
suggestions
Approach
• Quantitative
• Consistent and objective
• Individual sounds and
sentences are being
scored
• Qualitative
• Subjective
• Lecturer’s expertise and
perceptions plays a
crucial role
• Personal
• Provides overall score
Accessibility
• 24/7 free to access
• Allows learners to
receive feedback at any
time and anywhere
• Can handle many
students at the same time
• internet-based, can access
from various devices
• The availability is
limited and dependent
on the class schedule
Cost
• Offers 2 subscription
plans, standard
Rp379,000/month, and
Super plan
Rp1,839,000/year
• Looks free because they
only spend money for
the whole semester,
including all subjects
they attend
45
BoldVoice App
Lecturer OCF
Strengths
• 24/7 access
• Automatic
• Consistent
• Detail
• Emotional connection
• Motivational
• Trustworthy
• Can hear the voice
clearly rather than AI
Limitations
• Potential technical issues,
such as network and
device problems
• Lack of empathy
• Potential unreliability
• Awkward
• Makes student feel
nervous and afraid
• Incosistency
a. BoldVoice App Features Provided
After the researcher transformed each feedback provided into text
documents, the researcher analyzed it using text comparative analysis to see
each content that was provided from each feedback. The researcher found out
that AI feedback from BoldVoice App and lecturer oral corrective feedback
differ significantly in their structure, delivery, and overall approach.
The result shows BoldVoice App offers automatic feedback with
specific details strength and error of learners’ pronunciation performance,
including individual phonetic components such as vowels, consonants, and
combined sounds (diphthong). In addition to this, the researcher found that the
feedback is objective (unbiased) and displayed in a visual format such as using
graphs, charts, and color-coded systems. Furthermore, the scores given are
leveled, such as "Awesome," "Nice Job," "Good Effort," and "Bad," making it
easy for students to identify their strengths and weaknesses. It also gives scores
for individual sentences, providing a more detailed view of how learners are
doing on overall sentences not only on individual sounds of the words.
Those result is in line with Saraswati et al. (2023) and de la Vall & Araya
(2023) which shows that AI system provide immediate real-time feedback,
46
allowing learners to receive instant guidance on their pronunciation.
Furthermore, to supports the finding, the research from Zou et al. (2023) which
stated in Evenddy (2024) and research from Abernathy (2024) also found that
AI provide automatic feedback in language learning which offer a
comprehensive approach, such as personalized lesson and color graded system.
This structured format is designed to provide clear, objective, and consistent
feedback across multiple users, making it easy for learners to track their
progress over time and identify specific areas for improvement.
More analysis details, the researcher found out that the feedback from
BoldVoice App also includes AI Coach for teaching the user by using mouth
diagram (see Figure 6) or full video explanation to look how to pronounce
vowels and consonants correctly in American accent. This app gives feedback
not only by score but also coach’s sound vs test taker’s sound in pronouncing
the words and compared similar word sounds if the test taker mispronounced
it.
This result is aligned with Kurt (2022) research. The researcher uses
Speakometer as an English pronunciation AI and found out the automatic
feedback includes sounds comparisons between the user's pronunciation and a
model AI coach. Moreover, the result also confirmed by de la Vall & Araya
(2023). They stated that AI applications provides features which learners can
compare their pronunciation with native speaker model.
In addition, from the analysis on the app, the researcher found out that
the app is accessible 24/7. This flexibility ensures that it can help learners
eliminate waiting times to get the feedback due to class sizes. Moreover, this
app is accessible across various devices, including smartphones and tablets. It
gives them free to access any time and anywhere. This accessible support
students to practice and get feedback immediately.
This finding is nearly identical to the preview research entitled
Improving English Pronunciation with AI Speech-Recognition Technology.
47
According to Abimanto & Sumarsono (2024), AI is flexible and easy to access,
“they can use the app according to their own schedule.”
Nonetheless, the result also shows that all these features from BoldVoice
App come at a significant cost. Even though the user still can use BoldVoice
App, but it is limited to access all features. BoldVoice App offers plans ranging
from Rp1,390,000 per year to Rp1,839,000 per year.
This result in line to Nushi & Sadeghi (2021) which stated that
pronunciation app that using AI system such as Elsa Speak “offering in-app
purchases which provide you with a multitude of exercises and unlimited
practice but needs paying subscription” so learners should pay this kind of
subscription to assess all built-in features.
e. Lecturer Corrective Feedback Features
In contrast, according to the result of this study, lecturer feedback is
subjective, much more personal, conversational, and contextual.
This result in line to the research of Phuong (2022) which stated that
lecturer often provide feedback that is highly personalized, focusing on
individual errors rather than collective ones. Furthermore, the result is nearly
identical to Uludağ (2024) which stated that lecturer feedback is often related
by the context in which they teach. For instance, if the lecturer is teaching
online, lecturer should give their feedback strategies suited to an online
environment, which may differ from face-to-face settings.
Furthermore, unlike the algorithmic nature of BoldVoice App, this
study found that lecturer provides feedback considering student's background,
culture, and mother tongue influences. For example, it happened to Venus
that Betawi dialect affect her English pronunciation when she pronounced /r/
in words like "over” and it also happened to Mars which she difficult
pronounced sound /ʃ/ because in Bahasa Indonesia we do not have sound
similar to /ʃ/.
48
This result is confirmed by Ha et al. (2021). They found that teacher
corrective feedback has a relationship to sociocultural, contextual factors, and
students’ experiences. Sociocultural often includes background and native
language. Furthermore, the influence of mother tongue on pronunciation
cannot be overlooked. It also supported by the research of Kosasih (2021).
She identifies that students’ native language is a primary interference that
affecting students' pronunciation accuracy. Thus, the teacher should give the
feedback considering learners' linguistic backgrounds (Kosasih, 2021). This
highlights the importance of tailoring feedback to address individual learning
styles and backgrounds to ensure that it resonates with students' unique
experiences and challenges.
Moreover, the results from in-depth interview, as Earth was
mentioned “…. the lecturer asked me to read some words on the white board
and and then the lectures gave me the feedback from what I've read. Like she
told me what words are wrong and explained how to say it correctly.” The
researcher found out that lecturer feedback is delivered in real time during
classroom activities or one-on-one sessions. This indicates that lecturer
corrective feedback is given on the spot rather than delayed. Thus, if students
want to receive immediate correction and guidance, they have to attend the
class.
It is in line with the finding of Phuong (2022) which found that
teachers were found to mainly focus on instant feedback rather than delayed
feedback. This nature of instant feedback encourages students to remain
engaged and actively participate in the learning process.
Additionally, the researcher found from the interview that the
qualitative nature of lecturer feedback allows students to get deeper insight
into not just phonetic issues, but also broader linguistic challenges such as
intonation, rhythm, and fluency. It was shown to the feedback of Jupiter, the
lecturer said “…. however, the fluence, it needs to be fix ….”
49
It is in line with Sakale (2019), he stated that teacher feedback also
extends to fluency, including intonation and rhythm, which are critical
components of effective communication. Feedback that focuses on these
aspects helps students understand the nuances of language, such as stress
patterns and pitch variations. It is essential for conveying meaning and
emotion.
Another result from this study is that the lecturer can also consider
external factors such as nervousness that may affect a student’s
pronunciation. Moreover, lecturer often incorporate motivation into their
feedback to encourage students to overcome their difficulties and highlight
their progress in a more holistic manner.
It is in line with Martin et al. (2022), feedback that includes emotional
support, such as empathy and encouragement, can foster a positive learning
environment. This helps students feel motivated and comfortable with the
lecturer. Martin et al. emphasized feedback that is emotionally supportive,
clear, and engaging can help reduce student anxiety and make teacher-student
relationship closer.
Moreover, the result of this study found that traditional feedback is
unlike AI platforms. Traditional feedback typically happens in scheduled
sessions. It makes the accessibility limit to the class schedule and the
availability of the instructor. This requires students to attend classes or
individual sessions to receive feedback, which can be less flexible than
BoldVoice App.
This finding supported with the research result from Phuong (2022)
which found that teachers were found to mainly focus on instant feedback
rather than delayed feedback. This implies that when the student want to get
oral corrective feedback from the teacher, they have to attend the class
because the teacher mostly delivered the feedback immediately in the
classroom setting.
50
Despite these differences, BoldVoice App and teacher feedback share
common goals in improving students’ pronunciation. Both feedback
mechanisms aim to identify students’ strengths and weaknesses to guide them
to improve their pronunciation. While AI feedback provides detailed such as
specific phonetic analysis, teacher feedback offers an approach which
addresses not only pronunciation errors but also the student's overall
communication skills. AI feedback may be faster and more consistent,
especially for large groups of learners, but teacher feedback is more adaptable
to individual needs. It offers nuanced that are difficult for AI systems to
replicate.
2) AI Can Replace Lecturer Role or Be Complementary Tool?
Based on semi-structured interviews with the students, this study found out
that AI cannot fully replace human assessor in giving pronunciation feedback. For
instance, as Venus, Earth, and Jupiter said:
“Actually, do not fully agree, because in my own opinion Feedback from
a lecturer is also needed because a lecturer also has a sense of empathy
and that makes it different from AI. Although AI is also made to
encourage us better but the direct corrective feedback from lecturer in
my opinion affects the enthusiasm for students or pronunciation learners
in improving their pronunciatio. So, it’s 50:50.” – Venus
“Personally, I don't agree .... because like I said before, the importance
of interaction between lecturer and student. So, if it's with a robot (AI),
it's awkward, too quiet, and lonely. Language is an interaction between
humans because humans are social creatures. It’s verry different with
AI.” – Jupiter
“In my opinion, I don't agree. The problem is, AI is a program, a
program that may be designed in such a way to achieve perfection in an
aspect, for example in EFL itself … but of course there are many
51
shortcomings that are differs from the human side. Us, as humans, we
have emotions, we have empathy, which basically means we have a heart,
and AI doesn't have that. We humans have tolerance. It depends on the
condition of each student ….” – Neptune
This research result aligned with Lima & Wallace (2024) which stated
that AI systems on ChatGPT does not have emotional intelligence which can
encourage students to boost their confidence. They expressed that “ChatGPT,
while conversational and interactive up to a point, cannot match the dynamic,
real-time adaptation and emotional responsiveness of a lecturer. It cannot, for
instance, detect subtle cues in a student's speech that might indicate their
uncertainty about what they are saying or provide immediate, tailored
encouragement to boost a learner's confidence.” This limitation can affect
student motivation and emotional engagement, because AI cannot fully replicate
the touch of human feedback. Additionally, Deshmukh et al. (2008) support this
result by stating that the complexity of human language and the subjective nature
of pronunciation make it challenging for AI to fully mimic human assessment.
From the in-depth interview, the researcher also found that AI has its
disadvantages on technical barriers, such as device limitations and the internet.
“.... The weakness is of course the devices must support the app,
especially network problems. If there is a problem, it will affect the
program's performance.” – Venus
“Technology needs a proper device so that the results can be optimal.
For example, if we learn through BoldVoice, then if our smartphone is
not working, such as the microphone on the phone is broken or something
like that, it will be automatically affected .... and AI has a programmed
accent, for example, only British English.” – Neptune
This result supported by Shi & Deng (2024). They confirmed that most AI
generated feedback requires internet. The internet enables the instantaneous
52
delivery of feedback to students, which is crucial for timing of the feedback.
Studies have shown that optimal feedback timing, facilitated by internet
connectivity, can significantly impact user satisfaction and learning outcomes.
Furthermore, this study found out that AI-driven in English pronunciation
context might cannot capture differ accent and dialect. Most of the AI
pronunciation applications only offer American or British accent. Such as
BoldVoice App, it is only using American accent, which is it can affect scoring
and feedback while the test taker using differ accent.
This result is supported by Wang et al., (2023). This study found one of
the main challenges that may be faced by AI applications, they are struggle with
understanding diverse accents and dialects. It can make the feedback given and
learning outcomes for students not optimal. On the other side, lecturer can
capture the different English accents and can encourage learners to focus on only
one accent.
Besides the result that is already discussed above, not only AI App
(BoldVoice) that has its limitation, but lecturer corrective feedback also has its
weaknesses, for instance during pronunciation assessment with lecturer, the
student sometimes nervous and afraid. This was expressed by Venus and Earth:
“.... sometimes with the lecturer there is also a sense of nervousness
because we face-to-face with the lecturer ….” – Venus
"…. and I feel afraid too …." – Earth
This result is confirmed by Aprilia (2020) and Pratama et al. (2023)
which stated the presence of a lecturer, especially one who is a native speaker
can make learners afraid and it can heighten their anxiety. Students may fear
negative judgment or misunderstanding, which can make them nervous.
Thus, the researcher found the result from in-depth interview that all
students agree and suggest integrating AI in pronunciation classroom. They
53
mentioned AI and lecturer might be an effective approach in pronunciation
classroom, for instance, it can offer a synergistic approach where both the
strengths of AI and human-led feedback are leveraged. AI can serve as a support
tool which can provide detailed analysis and offer pronunciation practice outside
classroom hours. On the other hand, lecturer can review AI-generated reports and
address more complex issues that AI might miss. For example, AI can be used
for consistent practice and offering learners immediate feedback. On the other
hand, lecturer can interpret these AI results by providing deeper insights based
on individual student progress. Also, lecturer can give encouragement to motivate
them when necessary. This combination provides both precision and empathy to
enhance the learning experience.
This result is confirmed by Babaeian (2023). He stated that the
integration between AI feedback and teacher evaluations in collaborative
assessment models can enhance the overall effectiveness of pronunciation
assessment. By combining AI's automatic feedback features such as
accessibility, scalability, and consistency with the human touch of teacher
evaluations, learners can get the benefit from this comprehensive assessment
approach. This collaboration can offer a more holistic evaluation, which can gain
their learning experience and outcomes in pronunciation.
54
CHAPTER 5
CONCLUSION & SUGGESTIONS
A. Conclusion
This study explored and compared to identify the key differences and
similarities in the feedback provided by lecturer and BoldVoice App as an AI tool
included. This study comparing the criteria, depth, and quality of feedback.
Furthermore, AI feedback (re: BoldVoice App) and lecturer feedback in
pronunciation learning offer distinct advantages that complement each other in the
language learning process. AI feedback is characterized by its structured which
providing detailed phonetic analysis with consistency and immediacy. It is available
for 24/7, thus this AI support self-paced learning and offers objective assessments.
On the other hand, lecturer feedback is highly personal, conversational, and
contextual. This type of feedback addressed broader linguistic challenges such as
emotional intelligence and cultural backgrounds. Additionally, lecturer can adapt
their approach to learners individual learning styles and needs, offering a more
holistic learning experience.
Both approaches share the common goal of improving students'
pronunciation and encouraging self-improvement. While AI excels in providing
precise and technical feedback, lecturer offer invaluable insights into the nuances of
language use and communication. This suggests that an ideal solution might be a
hybrid model, combining the technological accuracy and consistency of AI with the
empathy and personal guidance of human lecturer. Moreover, as AI technology
continues to evolve, we may see more adaptive and personalized features
incorporated into AI feedback systems. However, the unique human elements that
lecturer bring to the learning process, such as emotional support, cultural sensitivity,
and the ability to address complex communication challenges will likely remain
irreplaceable.
In conclusion, the future of pronunciation feedback may lie in integrating
these approaches. By leveraging both the data-driven insights of AI and the nuanced
personal guidance of lecturer, students can receive comprehensive support in their
55
language learning journey. This combined approach might has the potential to offer
learners the most effective tools for improving their pronunciation.
B. Suggestion
The researcher suggests for other researchers to conduct research which
comparing AI-powered feedback with human assessor feedback over an extended
period (>1 years) to provide valuable insights into which methods lead to the most
significant long-term improvements. This study would track pronunciation progress
using standardized tests and expert evaluations, considering factors like error types,
learner backgrounds, and motivation levels.
Furthermore, the researcher suggests researchers to investigates effectiveness
of a hybrid model combining AI and human feedback. This approach might
potentially offer the best of them, which is combining the precision and availability
of AI with the touch and personal guidance of lecturer. Additionally, further research
focusing on lecturer/teacher perspectives is essential to conduct. This involves
exploring lecturer/teachers' current knowledge and perceptions of AI in language
learning, their willingness to incorporate AI tools, and any concerns about changing
roles. It is important to gather ideas from lecturer/teacher on how AI could
complement their teaching methods.
C. Implication
For teachers, the integration of AI pronunciation feedback in language
classrooms has significant implications, both in terms of challenges and
opportunities. These implications span various aspects of teaching, from lesson
planning to student assessment, and even the evolving role of the teacher in the digital
era. One of the primary implications is a shift in the teacher's role. With AI handling
much of the routine feedback on pronunciation, teachers may find themselves
evolving from being the primary source of feedback to becoming facilitators and
interpreters of AI-generated insights.
56
Furthermore, AI tools enable more personalized learning by providing
detailed student-specific data. This allows teachers to tailor their approach to each
student's needs. However, it also implies a need for teachers to become adept at
analyzing and acting upon this data effectively. Moreover, time management is
another significant implication. While AI can handle routine feedback tasks, teachers
may need to allocate time for understanding and interpreting AI data, as well as
explaining it to students. Initially, this might increase workload, but over time it could
free up time for more targeted instruction and higher-level pronunciation
development.
Thus, teachers will need to stay up to date with AI technologies and best
practices in their implementation. Teachers need to balance AI with their own
qualitative assessments that consider factors AI might miss. Additionally, classroom
dynamics may also change. With AI handling much of the basic feedback, classroom
time could be used more for application, discussion, and addressing higher-order
aspects of language learning that AI cannot cover effectively. To add, teachers need
to be aware of and address potential biases in AI systems. They need to ensure a fair
treatment of students from diverse linguistic backgrounds. This includes being
transparent about the limitations of AI.
Lastly, the availability of AI feedback may influence curriculum design. By
understanding and proactively addressing these implications, teachers can effectively
utilize the power of AI to enhance their teaching and to maintain the irreplaceable
human elements of education. The key is to view AI as a powerful tool that
complements not to replace the teacher's expertise and personal connection with
students.
For students, AI pronunciation feedback offers immediate, 24/7 access to
practice and improvement opportunities. This can lead to a more personalized and
flexible learning experience, potentially reducing speaking anxiety and increasing
learner autonomy. Students can get benefit from objective assessment and detailed
analysis of their pronunciation. It is often including visual representations of their
speech patterns. However, they need to be aware of AI's limitations in capturing
57
cultural and contextual nuances, the importance of balancing AI practice with human
interaction, and privacy considerations regarding their voice data.
58
REFERENCES
Abdurahimova, D. A. qizi, & Kobilova, N. R. (2022). Pronunciation problems of
English learners. 344–349. https://doi.org/10.24412/2181-2454-2022-1-344-349
Abernathy, P. C. (2024). AI tools as supplementary support in language acquisition AI
tools as supplementary support in language acquisition.
https://doi.org/10.20944/preprints202407.2213.v1
Abimanto, D., & Sumarsono, W. (2024). Improving English pronunciation with AI
speech-recognition technology. Acitya: Journal of Teaching and Education, 6(1),
146–156. https://doi.org/10.30650/ajte.v6i1.3810
Ani, A. (2019). Positive feedback improves students psychologiscal and physical
learning outcome. Indonesian Journal of Educational Studies (Ijes), 22(2), 144–
152.
Aprilia, W. (2020). English department students’ anxiety in interacting orally with an
English native speaker lecturer. Jambura Journal of English Teaching and
Literature, 1(1), 53–61. https://doi.org/10.37905/jetl.v1i1.5791
Asnawi., & Wariyati. (2021). The impact of using feedback to increase students’
motivation at the University of Muslim Nusantara Al Washliyah Medan. Research,
Society and Development, 10(1), e38210111784. https://doi.org/10.33448/rsd-
v10i1.11784
Babaeian, A. (2023). Pronunciation assessment: Traditional vs modern modes. Journal
of Education For Sustainable Innovation, 1(1), 61–68.
https://doi.org/10.56916/jesi.v1i1.530
Baker, A., & Burri, M. (2016). Feedback on second language pronunciation: A case
study of EAP teachers’ beliefs and practices. Australian Journal of Teacher
Education, 41(6), 1–19. https://doi.org/10.14221/ajte.2016v41n6.1
Bulut, O., & Wongvorachan, T. (2022). Feedback generation through artificial
intelligence. The Open/Technology in Education, Society, and Scholarship
59
Association Conference, 2(1), 1–9. https://doi.org/10.18357/otessac.2022.2.1.125
Cámara-Arenas, E., Tejedor-García, C., Tomas-Vázquez, C. J., & Escudero-Mancebo,
D. (2023). Automatic pronunciation assessment vs. automatic speech recognition:
A study of conflicting conditions for L2-English. Language Learning &
Technology, 2023(1), 1–19. https://hdl.handle.net/10125/73512
Coccia, M., & Benati, I. (2018). Comparative Studies. In Global Encyclopedia of Public
Administration, Public Policy, and Governance (pp. 1–7). Springer International
Publishing. https://doi.org/10.1007/978-3-319-31816-5_1197-1
Couper, G. (2016). Teacher cognition of pronunciation teaching amongst English
language teachers in Uruguay. Journal of Second Language Pronunciation, 2(1),
29–55. https://doi.org/10.1075/jslp.2.1.02cou
Couper, G. (2019). Teachers’ cognitions of corrective feedback on pronunciation: Their
beliefs, perceptions and practices. System, 84, 41–52.
https://doi.org/10.1016/j.system.2019.04.003
Creswell, J. W. (2018). Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches. In SAGE Open (Vol. 3rd). https://doi.org/10.4324/9780203803448-9
Derwing, T. M., & Munro, M. J. (2005). Second language accent and pronunciation
teaching: A research-based approach. TESOL Quarterly, 39(3), 379.
https://doi.org/10.2307/3588486
Deshmukh, O. D., Joshi, S., & Verma, A. (2008). Automatic pronunciation evaluation
and classification. Interspeech 2008, 1721–1724.
https://doi.org/10.21437/Interspeech.2008-464
Dewa, A. N., & Laksman-Huntley, M. (2022). Strategies to develop French
pronunciation using virelangue. Diksi, 30(2), 99–108.
https://doi.org/10.21831/diksi.v30i2.46770
Dey, I. (2003). Qualitative Data Analysis. Routledge.
https://doi.org/10.4324/9780203412497
60
Fathonah, M. B., Astrid, A., & Desvitasari, D. (2023). Speaking skills of EFL student
teachers of UIN Raden Fatah Palembang: Problems and strategies. IDEAS: Journal
on English Language Teaching and Learning, Linguistics and Literature, 11(1),
13–25. https://doi.org/10.24256/ideas.v11i1.3469
Fudholi, D., & Suominen, H. (2018). The importance of recommender and feedback
features in a pronunciation learning aid. Proceedings of the 5th Workshop on
Natural Language Processing Techniques for Educational Applications, 83–87.
https://doi.org/10.18653/v1/W18-3711
Graham, C., Buttery, P., & Nolan, F. (2016). Vowel characteristics in the assessment of
L2 English pronunciation. Proceedings of the Annual Conference of the
International Speech Communication Association, INTERSPEECH, 08-12-Sept,
1127–1131. https://doi.org/10.21437/Interspeech.2016-1630
Ha, X. Van, Nguyen, L. T., & Hung, B. P. (2021). Oral corrective feedback in English
as a foreign language classrooms: A teaching and learning perspective. Heliyon,
7(7), e07550. https://doi.org/10.1016/j.heliyon.2021.e07550
Harding, L. (2012). Pronunciation assessment. The Encyclopedia of Applied Linguistics.
https://doi.org/10.1002/9781405198431.wbeal0966
Haryanto, E. (2015). Teachers’ corrective feedback on students’ pronunciation at the
daffodils English Course Kampung Inggris Pare Indonesia. Linguists : Journal Of
Linguistics and Language Teaching, 2(2).
https://ejournal.iainbengkulu.ac.id/index.php/linguists/article/view/115
Herra, A., & Kulińska, A. (2018). The role of feedback in the process of learning
English as a foreign language. Forum Filologiczne Ateneum, 1(1(6)2018), 127–
143. https://doi.org/10.36575/2353-2912/1(6)2018.127
Iranifard, E., & Roudsari, R. L. (2022). Comparative research: An old yet unfamiliar
method (pp. 10(3): 3317-3318).
https://doi.org/https://doi.org/10.22038/jmrh.2022.66873.1954
Janesarvatan, F., & Asoodar, M. (2024). Constructive peer-feedback to improve
61
language and communication skills in medical education. Innovation in Language
Learning and Teaching, 1–15. https://doi.org/10.1080/17501229.2024.2311834
Kosasih, M. M. (2021). Factors affecting Indonesian students in learning English
pronunciation. International Research in Higher Education, 6(3), 13.
https://doi.org/10.5430/irhe.v6n3p13
Kurt, S. (2022). Speakometer: English pronunciation coach. Tesl-Ej, 26(1).
https://doi.org/10.55593/EJ.25101M2
Lee, A. H., & Lyster, R. (2016). The effects of corrective feedback on instructed L2
speech perception. Studies in Second Language Acquisition, 38(1), 35–64.
https://doi.org/10.1017/S0272263115000194
Li, Z., & Lian, A.-P. (2022). Achieving self-imitation for English intonation learning:
The role of corrective feedback. Chinese Journal of Applied Linguistics, 45(1),
106–125. https://doi.org/10.1515/CJAL-2022-0108
Lima, E. F., & Wallace, L. (2024). Friend or foe: Considering what pronunciation
teachers can do that chatgpt cannot. September 2023, 1–11.
https://doi.org/https://doi.org/10.31274/psllt.16111
Martin, S., Alvarez, I. M., & Espasa, A. (2022). Video feedback and foreign language
anxiety in online pronunciation tasks. International Journal of Educational
Technology in Higher Education, 19(1), 19. https://doi.org/10.1186/s41239-022-
00324-y
McCrocklin, S. (2019). Learners’ feedback regarding ASR-based dictation practice for
pronunciation learning. CALICO Journal, 36(2), 119–137.
https://doi.org/10.1558/cj.34738
Miles, M. B., & Hubermen, M. A. (1994). An Expanded Sourcebook: Qualitative Data
Analysis. Sage Publications.
Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24(4),
427–431. https://doi.org/10.1177/1362168820941288
62
Nushi, M., & Sadeghi, M. (2021). A critical review of ELSA: A pronunciation app.
Call-Ej, 22(3), 287–302.
Padilla, J. E., & Lee, K.-H. (2019). AI meets ESL: Assessing the effectiveness of AI-
Enabled speaking evaluations in English acquisition compared to traditional
techniques. International Journal of Advanced Scientific Research & Development.
(IJASRD), 1(3394), 7–20. https://doi.org/https://dx.doi.org/10.2139/ssrn.4581554
Pennington, M. C., & Rogerson-Revell, P. (2019). Assessing Pronunciation. In: English
Pronunciation Teaching and Research. In Research and Practice in Applied
Linguistics (pp. 287–342). Palgrave Macmillan. https://doi.org/10.1057/978-1-137-
47677-7_6
Phuong, T. T. H. (2022). Feedback on pronunciation: Vietnamese EFL teachers’ beliefs
and practice. SAGE Open, 12(3), 215824402211266.
https://doi.org/10.1177/21582440221126612
Pratama, W. R., Sabarun, & Qamariah, Z. (2023). Speaking test anxiety of the students
in the English study program of IAIN palangka raya. PUSTAKA: Jurnal Bahasa
Dan Pendidikan, 3(2), 1–16. https://doi.org/10.56910/pustaka.v3i2.414
Rahman, F. A., Kahfi, E. H., & Dalimunthe, R. N. R.-P. (2020). Exploring the
implementation of teacher’s corrective feedback on students’ pronunciation: A
case study in an Indonesian public high school. Beltic 2018, 477–485.
https://doi.org/10.5220/0008220804770485
Ramani, S., Könings, K. D., Ginsburg, S., & van der Vleuten, C. P. (2019). Feedback
redefined: Principles and practice. Journal of General Internal Medicine, 34(5),
744–749. https://doi.org/10.1007/s11606-019-04874-2
Rebolledo Font de la Vall, R., & González Araya, F. (2023). Exploring the benefits and
challenges of AI-Language learning tools. International Journal of Social Sciences
and Humanities Invention, 10(01), 7569–7576.
https://doi.org/10.18535/ijsshi/v10i01.02
Riswanto, R. (2022). The interference of EFL student’s native language in students
63
english pronunciation. JRTI (Jurnal Riset Tindakan Indonesia), 7(3), 357.
https://doi.org/10.29210/30031957000
Roothooft, H. (2014). The relationship between adult EFL teachers’ oral feedback
practices and their beliefs. System, 46, 65–79.
https://doi.org/10.1016/j.system.2014.07.012
Sadji Evenddy, S. (2024). Investigating AI’s automated feedback in English language
learning. FLIP Foreign Language Instruction Probe, 3(1), 76–87.
https://doi.org/10.54213/flip.v3i1.401
Saeli, H., Rahmati, P., & Dalman, M. (2021). Oral corrective feedback on pronunciation
errors: The mediating effects of learners’ engagement with feedback. Advances in
Language and Literary Studies, 12(4), 68.
https://doi.org/10.7575/aiac.alls.v.12n.4.p.68
Saito, K. (2021). Effects of corrective feedback on second language pronunciation
development. In The Cambridge Handbook of Corrective Feedback in Second
Language Learning and Teaching (pp. 407–428). Cambridge University Press.
https://doi.org/10.1017/9781108589789.020
Sakale, S. (2019). The important role of teachers’ feedback during speaking sctivities in
Moroccan classes. Arab World English Journal, 10(3), 344–351.
https://doi.org/10.24093/awej/vol10no3.24
Saraswati, G. P. D., Farida, A. N., & Yuliati, Y. (2023). Implementing AI Replika in
higher education speaking classes: Benefits and challenges. ELT Forum: Journal
of English Language Teaching, 12(3), 207–215.
https://doi.org/10.15294/elt.v12i3.76525
Shi, Y., & Deng, B. (2024). Finding the sweet spot: Exploring the optimal
communication delay for AI feedback tools. Information Processing &
Management, 61(2), 103572. https://doi.org/10.1016/j.ipm.2023.103572
Syed, F., & Hussein Abdelrady, A. (2022). Pronunciation Problems Encountered by
EFL Learners: An Empirical Study. SSRN Electronic Journal, 12(December), 194–
64
212. https://doi.org/10.2139/ssrn.3998342
Takidze, M. (2024). Significance of teaching and learning English phonetics
(Pronunciation), influencing factors and the methods to overcome those challenges.
Enadakultura. https://doi.org/10.52340/lac.2024.31.20
Trofimovich, P., & Isaacs, T. (2016). Second language pronunciation assessment: A
look at the present and the future. Second Language Pronunciation Assessment:
Interdisciplinary Perspectives, 259–271. https://doi.org/10.21832/ISAACS6848
Uludağ, O. (2024). Exploring the relationship between teachers’ beliefs and practices of
oral corrective feedback in synchronous computer-mediated communication.
Heliyon, 10(8). https://doi.org/10.1016/j.heliyon.2024.e29507
Umar, R. L. U., & Aspany, N. F. (2024). Students’ pronunciation skill on the ability of
suprasegmental and segmental aspects in English pronunciation. NUSRA: Jurnal
Penelitian Dan Ilmu Pendidikan, 5(1), 314–319.
https://doi.org/10.55681/nusra.v5i1.2196
Umarella, R. F. (2023). Students’ Perception on Teacher’s Feedback in Speaking Class
Rahmatul. Proceedings of the 69th TEFLIN International Conference in
Conjunction with the 3rd English Education International Conference (EEIC), 40–
47.
Wang, T. (2023). Factors affecting second language pronunciation acquisition.
Frontiers in Educational Research, 6(20).
https://doi.org/10.25236/FER.2023.062003
Wang, T., Lund, B. D., Marengo, A., Pagano, A., Mannuru, N. R., Teel, Z. A., & Pange,
J. (2023). Exploring the potential impact of artificial intelligence (AI) on
international students in higher education: Generative AI, Chatbots, Analytics, and
International tudent success. Applied Sciences, 13(11), 6716.
https://doi.org/10.3390/app13116716
Zou, B., Du, Y., Wang, Z., Chen, J., & Zhang, W. (2023). An investigation into
artificial intelligence speech evaluation programs with automatic feedback for
65
developing EFL learners’ speaking skills. SAGE Open, 13(3).
https://doi.org/10.1177/21582440231193818
66
APPENDICES
APPENDICES
67
Appendix 1: Surat Izin Penelitian
68
Appendix 2: Surat Tugas Validator
69
Appendix 3: Blueprint of Semi-structured Interview
Indicator
Question
The Role of Feedback in
Language Learning
Could you describe your understanding of the role and
importance of feedback in the learning process?
The Importance of Feedback
on Pronunciation
How important is feedback for learners’ pronunciation
ability?
The Impact of Feedback on
Pronunciation
How do you believe feedback impacts learners’
pronunciation in language learning?
Whether feedback can engage students to improve their
pronunciation ability and does the teacher interrupt the
student during student’s pronunciation performance to
correct the pronunciation?
Types of Pronunciation
Feedback
What kind of feedback that can help learners improved the
English pronunciation? Please give the reason!
Have you ever got feedback from lecturer on your
pronunciation in classroom setting? If you have, can you
share your personal experiences regarding the feedback?
What kind of feedback from the lecturer that you think will
help your pronunciation? Do you prefer it in formal or
informal way? Written or spoken feedback?
AI on Pronunciation Practice
Has your lecturer ever used AI in the pronunciation
classroom? If it is no, have you use AI to practice or assess
your English pronunciation?
Comparing AI Feedback and
Lecturers' Direct Corrective
Feedback
In your opinion, what are the strengths and limitations of
feedback provided by artificial intelligence (AI) compared
to feedback given by a human teacher? How do you prefer
to receive feedback, and why?
AI Replacement Role
Do you agree that AI can replace human teacher on giving
feedback in EFL setting?
Adapted from:
Umarella, R. F. (2023). Students’ Perception on Teacher’s Feedback in Speaking Class
Rahmatul. Proceedings of the 69th TEFLIN International Conference in
Conjunction with the 3rd English Education International Conference (EEIC), 40–
47.
70
Appendix 4: Linguaskill Speaking Phonological Descriptors for Read-aloud
Overall Intelligibility
Individual Sounds
Stress, Rhythm and
Intonation
Mark
Pronunciation is easy to
understand and meaning is
conveyed effectively.
Individual sounds are clear
and unambiguous.
Stress, rhythm and intonation
are consistently used
appropriately so that meaning is
expressed effectively.
6.0
Some features of 5 and some features of 6 in approximately equal measure.
5.5
Pronunciation is easy to
understand and meaning is
conveyed well.
Individual sounds are
generally clear and
unambiguous.
Stress, rhythm and intonation
are generally used appropriately
so that meaning is expressed
well.
5.0
Some features of 4 and some features of 5 in approximately equal measure.
4.5
Pronunciation can generally
be understood and meaning is
conveyed adequately.
Individual sounds are
generally clear although
there may be occasional
difficulty for the listener.
Stress, rhythm and intonation
are used to express meaning
adequately.
4.0
Some features of 3 and some features of 4 in approximately equal measure.
3.5
Pronunciation can generally
be understood but L1 features
may cause strain; meaning is
conveyed but there may be
some ambiguity.
Many individual sounds
are clear but some may
cause difficulty for the
listener.
An attempt is made to use
aspects of stress, rhythm and
intonation to express meaning.
3.0
Some features of 2 and some features of 3 in approximately equal measure.
2.5
Pronunciation of single words
may be intelligible but L1
features may make
understanding difficult and
some meaning may be
distorted.
Inaccuracies in the
pronunciation of individual
sounds may cause strain
for the listener and may
impede communication of
meaning.
Little attempt is made to use
aspects of stress, rhythm and
intonation to express meaning.
2.0
Some features of 1 and some features of 2 in approximately equal measure.
1.5
Pronunciation of single words
may be intelligible but L1
features may cause excessive
strain to a listener and
meaning may be seriously
distorted.
Serious inaccuracies in the
pronunciation of individual
sounds may make speech
unintelligible.
No attempt is made to use
aspects of stress, rhythm and
intonation to express meaning.
1.0
Throughout the task, the response(s) are not attempted or consistently no meaning is conveyed.
0
71
Appendix 5: Pronunciation Assessment: Read Aloud rubric from Khon Kaen
University
Excellent
(2.5 pts)
Good
(2 pts)
Fair
(1.5pts)
Poor
(1 pt)
Pronunciation,
clarity
Pronounces all
words correctly
and speaks
clearly.
Pronounces most
of the words
correctly and
speaks clearly.
Speaks clearly;
mispronounces
some words.
Difficult to
understand, is
struggling or
mispronounces
most words.
(4-9 errors)
(10-17)
(18+)
Fluidity
Reads at an
appropriate
speed. No
lapses,
hesitation or
wavering in the
pace.
Punctuation
pauses are
present
Reads at an
appropriate speed.
Few lapses,
hesitation or
wavering
Punctuation
pauses are present
most of the time
Reads at an
appropriate
speed most of
the time. Some
lapses,
hesitation or
wavering.
Punctuation
pauses are
present some of
the time
Rarely keeps the
appropriate
speed. Many
lapses,
hesitation or
wavering.
Punctuation
pauses not are
present
(4-5 errors)
(6-9)
(10+)
Volume
The intensity of
the voice is
clear and
audible all
through the
reading. No
fluctuations are
perceived
The intensity is
clear and audible
most of the time.
The intensity is
somewhat clear
and audible but
some words or
sentences are
barely
imperceptible
The volume is
not in the
audible range,
many words or
sentences are
barely
imperceptible
(4-5 errors)
(6-9)
(10+)
Intonation,
expression
The student
keeps the
required tone.
Does not
overdo it or
exaggerate.
Does not sound
monotone or
artificial
Adequate tone
most of the time.
Does not overdo
it or exaggerate.
Does not sound
monotone or
artificial
Adequate tone
some of the
time.
Occasionally
does sound
monotone,
artificial or
exaggerated
Rarely adequate
tone
Does sound
monotone,
artificial or
exaggerated
72
Appendix 6: Lecturer’s Feedback Transcript
(Venus)
Dosen: Oke ya thank you Venus overall sudah oke ya. Hanya saja tadi sound /θ/ nya ya
masih ketuker-tuker. Tapi sebetulnya voice semua jadinya. Kayak /boʊθ/ itu sebetulnya
kan harusnya voiceless ya /ˈhel.θi/ /ˈθæŋk ˌjuː/ itu juga harusnya voiceless tapi masih
masih kurang tipis ya. Oke “they were looking for a good book” number 1 oke.
Number 2, tadi masih long vowelnya kurang, “I made three” ya, /θriː/, keluarnya masih
/tri:/ ya. Ini sound /θ/ nya belum keluar. “sugar cookies and a great fig cake” long vowel
/keɪk/
No. 3, “new theater” /ˈθiː.ə.tɚ/ “student over there”.
kamu orang mana?
Venus: Orang Betawi
Dosen: Orang betawi jadi (kata) “over” nya ini (sound) /r/ nya terlalu ini ya terlalu betawi
“overr” seharusnya “ōvər ðer” oke /ð/ juga sudah
(number 4 kata) “push” nya sudah oke, “through” nya yang ke ini ya, /th/ (pada) /θruː/
voice or voiceless?
Venus: Voiceless
Dosen: Voiceless ya. “true” jadi keluarnya, (sound) /t/ jadi keluarnya.
(Number 5) “She is super thankful for the beautiful birthday flowers” ya tadi Anda
keluarnya “birday” /th/ nya hilang. /ˈbɝːθ.deɪ/ jadi /t/ sama /d/ jadi /ˈbɝːθ.deɪ/ ya. Nah
/ˈflaʊ.ɚs/ ya bukan /ˈflow.ɚs/ ya. kata “flowers” ini mispronunciation masih.
Number 6, “The days are so long and slow when you’re young” sudah oke.
Number 7, “did the” (itu) /ð/ ya, oke.
Number 8, oke.
73
Number 9, (kata) “those” /ðoʊz/ itu sudah betul ya tapi kurang, kurang long vowelnya.
/ðoʊz/ /ðiːz/, this and that bentuk jamaknya “these” /ðiːz/, “that” /ðoʊz/ ya dia pake long
vowelnya, oke. Number 10 oke ya. Sip! I'll give you I think 75.
(Mars)
Dosen: Good, Oke. Number 1, “They were looking for a good book” ini udah oke.
Number 2 “fig cake”. Ini masih di overall (the six student there) ini masih di sound /th/
nya ya. Jadi saya gagal ngajar /th/ nya. Emang udah kutukannya nggak bisa nyebut /th/
ya hahaha.
Ini masih agak jadi sebetulnya you already understand the konsep of voice and voiceless
ya cuman ketika anda ketemu dengan voiceless itu jadi /t/ keluarnya ya jadi /t/, gimana
sih caranya miss ya, ya kayak kata “truth” ya, tadi F2 masih bilang /truːt/. Ada /h/ nya,
tapi nggak terlalu. tipis ya. Kita udah latihan “through” loh kemarin, tapi masih kelewat.
oke number 6 ya, “the days” /ðə deɪs/ nah ini giliran yang voiceless. Itu bacanya sudah
lambat ya, gimana kalo 60 km/jam ya. “The days are so long and slow” /sloʊ/ itu /w/ nya
itu jadinya agak samar, /sloʊ/ ya. “slow when you’re young” /jʌŋ/ bukan /joŋ/.
Number 7 oke
Number 8 itu ketemu tadi kata “another” pakai voice atau voiceless?
All Students: Voice
Dosen: “another” bukan voiceless. kalo “healthy” baru ya oke. Kata “those” nya udah
oke, ini “things” nya ini tadi kaya “think”. “bathroom” juga sudah oke., oke
Nah mungkin ini nomor 9 ini banyak banget /th/ nya jadi agak-agak ribet
Number 10 nya udah oke ya, thank you oke I give you 80.
74
(Earth)
Dosen: Oke, thank you, so overall ini sudah understandable when you say into native ya.
However, some words that you need to pay attention especially for the short and long
vowels ya. ini masih apa masih ketemu kesalahannya sama tadi dengan F4 the word
“made” “cake” ya itu itu kan diphthong tuh /eɪ/ jadi harusnya dibacanya long ya.
Kemudian “push” ya, “push” diphthong nya apa? /ʃ/ ya. Tadi kurang kental ya, sound /ʃ/
nya ini belum masuk karena di Bahasa Indonesia ini gak ada kata /ʃ/. Kita tidak menyebut
kata “syaitan” tapi “setan” ya hahaha. Jadi “push” tadi kaya /pʊs/ kurang /ʃ/ nya, ya oke
gapapa.
“birthday” ini tadi /th/ nya juga masih, jadi /ˈbɝːθ.deɪ/ /ˈbɝːθ.deɪ/ sound /θ/ nya itu harus
voiceless banget ya.
Nomor 6 oke. Nomor 7 kata “shoes” tadi tuh. F3 berarti ada masalah di sound /ʃ/ ya.
Sound /ʃ/ itu diphthong /sh/ itu harusnya lebih di stressing lagi.
“daughter” itu /ˈdɔː.tɚ/ itu /ɔ/ ya bunyinya bukan /ˈdauː.tɚ/ “daughter is cooking another
healthy” ini oke.
Nomor 8, “daughter” masih. nomor 9 “those thing about the bathroom” ya “bedroom”
atau “bathroom”? apa F3?
Earth: “bathroom”
Dosen: /ˈbæθ.rʊm/ yes itu masih kelewat tadi oke
Nomor 10 “They both had an amazing” itu “had an” /ˈhæd.ənt/.
If you have difficulties to connect “had an” jadi /ˈhæd.ənt/, jadi lebih baik dikasih jeda
“had-an amazing view of today’s football game” /ɡeɪm/ ya. Kita ada sih kenal kata
“game” ya tapi nyebutnya jadi “gem” gitu tapi sebetulnya harusnya long vowel /ɡeɪm/ ya
jadi ada ada diphthong e dan i /ɡeɪm/ oke yap.
Nah ini sebetulnya mother tongue juga mempengaruhi ya karena kita tidak ketemu kata
/th/ tidak ketemu kata long vowel dan /sh/ itu juga bisa mempengaruhi pronunciation kita.
Oke, thank you very much I give you 80 yes.
75
(Jupiter)
Thank you, oke it sounds clearly ya when you read these sentences ya. however the
fluence ya it needs to be fix. Mungkin karena apa namanya jarang untuk membaca teks
dalam bahasa Inggris ya it's oke.
Nah things that I highlighted is the sound /t/ and /θ/ ya ini masih rata bunyinya semuanya
dibikin voice ya. padahal disitu seperti kata apa itu “bathroom” “both” harusnya pakai
voiceless, “thank” itu pakai voiceless itu saya tanda-tandain. Nah kalau /θ/ itu kan
harusnya dia voice ya /θ/, tapi karena dia voice sih sudah betul hanya saja keluarnya jadi
/de/. Nah, ini sebetulnya masih keliru. Untuk selanjutnya bisa diperbaiki. “There” itu juga
voice, harus bunyi, tapi keluarnya jadi /de/ sama jadi “I Made” /aɪ meɪd/ itu jadinya sama.
Beberapa words yang masih wrong pronouncing, misal kata “goes” anda masih pakai
/goos/ ya. nggak boleh nyontek hahaha. oke nah ketika ketemu dengan “those” and
“thing” itu kan voice and voiceless ya “those thing” nah ini bunyinya sama semua jadi
/th/ nya jadi /t/ ya apa lagi ya “birthday” /ˈbɝːθ.deɪ/ ya.
Oke untuk apa penggabungan itu sudah oke, ya. Hanya saja tadi yang “I made”, “made”
long vowels ya, /meɪd/. Jadi Anda tadi ketemunya /aɪ med/, “cookies” itu long vowels,
itu juga masih terlewat ya.
“Who’s the new theater student over there?” “there” /ðer/ itu /ð/ ya
“As the saying goes” /ɡoʊz/ “face the” /ðə/ nah itu juga masih.
Alright sudah saya komentarin “another” itu sudah “healthy” sudah ya jadi masih PR
besar di itu ya /th/ sounds. oke thank you.
76
(Saturn)
Ok, good thank you. Alright much better. Masih salah di pronunciation number 5 ya
/ˈflaʊ.ɚ/ ya “flowers” nya masih salah /ʃuːs/ nya masih salah sound /th/ nya masih salah
kurang tipis ya jadi keluarnya masih jadi /t/ “bathroom” /ˈbæθ.rʊm/ /hel.θi/ bukan /hel.ti/
ya.
Nomor 10 oke, ini kayaknya rata-rata nomor 10 bagus.
Oke, dari nomor 1, “for a good” kata “good” ini /ɡʊd/ long vowel, “good book” /ɡʊd
bʊk/.
Oke, “made” nya oke, “cake” nya oke, “over there” /ˈoʊ.vɚ ðer/ harus konsisten kalau
mau British semuanya ya jangan dicampur-campur Amerika oke.
“Super” /ˈsuː.pɚ/, not /super/. “You're young” kan ada “you” “are” ya jadi kalo dia
disatuin jadi /jʊr/ /jʊr jʌŋ/tadi kurang anda cuma bilang /yor yang/.
Right, done. Thank you. I give you 80%.
77
(Neptune)
Dosen: Oke thank you, so you good ya, you good on reading this. Why are you not doing
it good in the classroom? hahaha.
Neptune: Because I’m too nervous.
Dosen: Yes, too nervous. No worries. Alright ya em, jadi most of the word has been
pronounced quite well ya for the 10 sentences ya, however the sounds of some /r/ ya dan
/eh/ itu masih terlewat ya. “for a good” /fɔːr eɪ ɡʊd/ ya kemudian “she’s super” you made
2 /s/ there ya she tapi /s/ nya jadi hilang satu “she super” so “she’s super” ya and “you
are young” “you’re young” /jʊr jʌŋ/ itu juga masih ketindih jadi dengan kata yang
disingkat itu Anda masih perlu diperbaiki.
Saya menemukan bahwa apa namanya accent yang, words yang anda pronounced itu
sudah oke semua ya terutama di sounds /th/ dan /d/ nya itu sudah mulai bagus hanya saja
masih perlu latihan untuk yang voiceless nya ya kayak “three” “thing” “tough”
“bathroom” “both” itu kan voiceless ya itu masih harus ada tekanan sedikit.
Oke, that's all. Thank you F6.
Ya, lulus.
78
Appendix 7: Learners’ Interview Transcripts
(R = Researcher) (P1 = Venus)
R: Please explain your understanding of the role and importance of feedback in the
learning process especially learning English on pronunciation.
P1: In my opinion, the role and importance of feedback in the learning process is very
important because it will be one of the motivational values for students to be able to
improve our abilities.
R: How do you think feedback can affect the pronunciation of students, especially
yourself in learning English?
P1: For feedback on pronunciation itself, especially for us as English education students,
I think it is very much needed because it will also help students to realize their weaknesses
and strengths in learning pronunciation itself.
R: Can feedback make you engaged to improve English pronunciation and does the
lecturer in your class ever interrupt when you are practicing speaking in class to correct
your pronunciation?
P1: Well, yes it makes me engaged and about the intruption, nope she never. The lecturer
gives students the opportunity to try to pronounce the sentences provided, then the
lecturer will give feedback at the end after we tried to read it. If the student has a mistake,
then the lecturer gives the example how to pronounce it.
R: So the lecturer it's like giving the students opportunity to read first or to speak first and
then at the end give the feedback, right? Like telling you the wrong part is here.
P1: Yes, that's right.
R: What kind of feedback do you think can help you to improve your English
pronunciation?
79
P1: Feedback that directly show how to master a consonant or vowel itself. Because what
I feel at this time sometimes there is still mispronunciation that often occurs, and it is
often left unchecked. Therefore, the correction from the lecturer itself is also very
important. Then the reason is because it will affect the student to be able to learn also
outside of learning in class.
R: So do you prefer feedback that is directly provided after you pronounce?
P1: Yes, that’s right
R: Okay, thank you. Have you ever gotten feedback from your lecturer on your
pronunciation in class? If so, can you give a little experience?
P1: Okay, so with this pronunciation lecturer, especially with Miss X, there is always an
opportunity after Miss X explains the material. The students will be given the opportunity
who wants to try both in online and offline classes, then a student who tries, Miss X will
listens and corrects if there are any mistakes.
R: Thank you. What kind of feedback from lecturers do you think can help improve your
pronunciation and do you prefer the feedback to be formal or informal and written or
spoken?
P1: For feedback, it's more about the way it's delivered, like directly. So, I prefer the
formal correction. Also directly spoken. Because sometimes with oral feedback, I also
get a direct capture about my mistake. But, written ones can be a note to improve even
more.
R: That means with feedback like yesterday is what you want? Alright, does your lecturer
ever use an AI in pronunciation classes? If not, have you ever used AI to train or assess
your English pronunciation?
P1: For the lecturer herself in class has never used AI as far as I know but I have tried
several times use AI for practice speaking. But the speaking application itself is not very
helpful for pronunciation
80
R: In your opinion, what are the advantages and disadvantages of the feedback given by
AI compared to the feedback given by the lecturer directly? Do you prefer to receive
feedback from AI or lecturers and why?
P1: Okay, so for the advantages and disadvantages of AI, let's say that the advantages are
that we can access the application whenever we need it when we want to learn or have
free time and the advantages AI will record our voices in the application so it helps us to
see our level or our mistakes and strengths in Pronunciation. The disadvantages are of
course the devices needed must also support, especially network problems if there is a
disturbance that will affect the performance of the application. Then, compare with the
lecturer I sometimes feel nervous with the lecturer because it is face to face, and
sometimes nervousness can affects how we practice pronunciation in front of the lecturer.
R: Okay thank you, it means that from AI it doesn't cause nervousness because you can
access it alone, so no one is monitoring because only AI is assessing, while from the
lecturer the lecturer is face to face or even in class there are friends too. Do you agree that
AI can replace the role of teachers or lecturers in providing feedback, especially in the
EFL context?
P1: Actually, I don't agree but also don't fully agree, because in my own opinion, feedback
from a lecturer is also needed because a lecturer also has a sense of empathy and that's
what makes it different from AI. Although AI is also made to encourage us to be better
but the direct delivery of the lecturer in my opinion also affects the enthusiasm for
students or pronunciation learners in improving the quality of their pronunciation. so,
50:50
R: Then if it's 50:50, do you agree with combine? It’s like lecturer can use AI in learning
process especially for feedback aspect.
P1: I think it’s possible because by utilizing the application it also makes it easier for us
to track our pronunciation practice activities not only in class but also outside of class.
P1: Alright, thank you for your time
81
(R = Researcher) (P2 = Mars)
R: thank you for your time. Could you please explain your understanding of the role and
importance of feedback in the learning process?
P2: In my opinion, feedback has an important role in the learning process because it can
help students understand their strengths and weaknesses and can also provide direction
for improvement.
R: okay, thank you. How important is feedback to your pronunciation?
P2: For me, feedback isvery important because it can help correct mistakes and develop
fluency in speaking. Then without feedback maybe for example we have a mistake, but
we don't know where our mistakes are and maybe we will continue to say the wrong ones.
so, it is very, very important that feedback is needed.
R: How do you think feedback can affect the pronunciation of students in learning
languages, especially in English?
P2: I believe that feedback can affect students' pronunciation in learning especially in
English by giving proper corrections and giving clear examples.
R: Can feedback make students more engaged to keep improving their pronunciation?
And do lecturers ever interrupt when students are practicing speaking in class with the
purpose that the lecturer wants to ccorrect students’ pronunciation immediately?
P2: Yes, feedback can encourage students to improve their speaking skills. And I often
found the lecturer when I was speaking, the lecturer interrupted to correct my
pronunciation.
R: Is it only focuses on pronunciation class or in all classes that need to speak English?
only in pronunciation class or in other classes too?
P2: No, speaking too, pronunciation too, phonology too, grammar in grammar class it is
not in terms of pronunciation but sentence structure.
R: oh so, that means it was often interrupting you when you are practice speak in English?
Okay I got it. What kind of feedback do you think can help students to improve their
82
English pronunciation? please give the reasons. For example, like I prefer direct feedback
when I speak, or I prefer feedback that is just given notes by the lecturer.
P2: For me, feedback that directly identifies specific errors in pronunciation and gives
clear examples of correct pronunciation, like for example if I'm wrong, lecturer
immediately interrupt me to correct it.
R: oh directly interrupt, like yesterday what Mrs. X did to you? you guys read first, then
finish and then give feedback. means you prefer when you are reading or when you are
talking directly cut?
P2: I like direct feedback, so for example I repeat a sentence, so the previous sentence
that was wrong is immediately correct.
R: oh so, so you prefer that rather than waiting until it's finished and then correcting it.
okay thank you. have you ever gotten feedback from a lecturer on your pronunciation in
class? If so can you provide personal experience related to the feedback?
P2: Yes, I have had that experience in class. When I was did a dialog with my friend and
after it, then my lecturer immediately identified what was wrong in my pronunciation and
also my friend.
R: oh okay thank you, what kind of feedback from lecturers do you think can help improve
your pronunciation? Do you prefer formal or informal, written or oral corrective?
P2: Yes directly. I prefer oral feedback because it's easier to understand.
R: Does that mean directly interrupted or like the lecturer gave yesterday?
P2: yes, directly interrupted. prefer to be interrupted.
R: okay, has your lecturer ever used AI in pronunciation class? If not, have you ever used
AI to train or assess your English pronunciation?
P2: Never before, my lecturer never used AI in pronunciation class. However, I have used
AI for my English pronunciation practice and assessment.
R: Can you mention what the AI is?
83
P2: BoldVoice app
R: What do you think are the advantages and disadvantages of the feedback provided by
the AI compared to the feedback provided by the lecturer? And do you prefer to receive
feedback from AI or lecturers? And why?
P2: The advantages of AI is ability to provide feedback is consistent, fast, and the
disadvantages maybe that AI is still lacking in understanding the context and nuances
possessed by a human. so, it's like it's more awkward, and also the lack of AI is that we
have to use AI must use a proper device so it's not slow, then also the internet, the signal
must be good and must use enough internet quota, right. Then for lecturers, I prefer to
receive feedback from lecturers because it can provide a more personal and in-depth view
and also yesterday the lecturer gave feedback as well as providing motivation, so I can
feel the motivation rather than being motivated by AI.
R: But what do you think the disadvantages of the lecturer itself?
P2: sometimes the lecturer gives feedback, you are wrong here but doesn't immediately
correct it, maybe immediately correct it but sometimes it's not and sometimes we are
afraid too, face to face with them.
R: Do you agree that AI can replace the role of lecturers or teachers in giving feedback to
you?
P2: Personally, I don't fully agree that AI can completely replace teachers in providing
feedback in the context of learning English as a foreign language. maybe it can be mixed,
so we learn with lecturers and learn with AI too, but if to replace it, it can't even though
it can provide feedback, useful feedback, the presence of a human teacher is still
important to provide more in-depth and personalized guidance and also if the teacher it is
more have knowledge.
R: So you mean that if with a teacher or lecturer, as a human being, they can feel what
the student is feeling, they can give motivation too, give empathy too, so it's like they can
look condition the student's condition, also can see students’ environment, while this AI
can't, right?
84
P2: Yes, the lecturer also knows that our mother tongue is like this, so she understands
us, she can motivate us more.
R: oh, you right, and AI can't detect your mother tongue, and it is also only specific to
one accent, like yesterday your lecturer mentioned that it only focuses on American while
in learning like this any accent can happen. So, do you think it can be combined in
learning English pronunciation in class?
P2: maybe it can be. For example, in giving material we can use AI but giving feedback
I think it's more for the lecturer. the material can be from AI.
R: okay, okay, thank you for your time.
85
(R = Researcher) (P3 = Earth)
R: Could you please describe your understanding of the role and importance of feedback
in the learning process?
P3: OK. So feedback is like a helpful, helpful guide that tells us how well we are doing
in our learning process. When we receive feedback, we can see what we are doing well
and what we need to work on. And this helps us learn and grow because we can make
change to be even better. And feedback is important because it helps us understand where
we stand and how we can get better. So it's really important to pay attention to feedback
and use it to improve our learning.
R: OK, So what do you think? How important is feedback for learners' pronunciation
ability?
P3: Feedback is very important for helping learners improve their pronunciation. And
when we talk about pronunciation, we mean that how we say words and sounds in a
language. And getting feedback means getting information about how well we are
pronouncing words correctly.
R: OK, thank you. How do you believe that feedback can impact learners' pronunciation
in language learning?
P3: Feedback plays a crucial role in helping learners improve their pronunciation in
language learning. When learners have received feedback on their pronunciation, they
can better understand their mistakes and work towards correcting them. And for example,
if a learner mispronounced a word and received feedback on the correct pronunciations,
they can practice saying the word correctly until they improve.
R: OK, OK, I agree with you. Thank you. F3, whether feedback can engage students to
improve their speaking ability? And does the teacher or your lecturer interrupt you as a
student during a student speaking performance to correct the pronunciation in the
classroom?
P3: OK, so in my opinion it can engage students to improve their speaking ability because
by giving feedback, students are able to know about their speaking ability. And yeah, it
86
happens many times when the lecturer interrupts us when we speak in class, for example
my several lectures, they often interrupt us while practicing reading or speaking in the
class. And while we are speaking to correct us. This interruption is done to help us
improve our pronunciation and speaking more clearly.
R: OK, OK, What kind of feedback that can help learners improve their English
pronunciation? Please give the reason.
P3: OK, in my opinion, feedback that focuses on the sound they are struggling with. For
example, instead of just saying your pronunciation is bad, a more helpful feedback would
be to try to work on pronouncing the sound more clearly. So by providing specific
feedback on the sounds that need improvements, learners can focus on practicing those
particular areas.
R: So you prefer when the lecturer gives the feedback like oh you better in that sound. So
the lecturer gave the example of how to pronounce the words and how to pronounce
correct sentences or correct words directly, right?
P3: Yes, I prefer to. I prefer the teacher to interrupt us directly to correct. Yeah, to interrupt
us when we are talking and speaking.
R: not in writing feedback like giving notes?
P3: No, I don't like it.
R: OK. Thank you. Have you ever got feedback from the lecturer on your pronunciation
in a classroom setting? If you have, can you share your personal experiences regarding
the feedback?
P3: Yes, I've ever got the feedback from my lecturer when I learned. For example, when
I learned in class and especially in pronunciation subject, the lecturer asked me to read
some words on the white board and and then the lectures gave me the feedback from what
I've read. Like she told me what words are wrong and explained how to say it correctly.
R: So the lecturer gave the text?
87
P3: Sentences in the powerpoint. And we read it and after that the lecture will interrupt
us. Where's wrong? Where's the word that is pronounced wrong.
R: Oh OK OK got it. Thank you. What kind of feedback from the lecturer that you think
will help your pronunciation? Do you prefer in a formal or informal way and written or
spoken feedback?
P3: I prefer an informal way, and also spoken ones, because I think it's going to be easier
for me to listen to the explanation of the lecture directly.
R: OK, so you can directly listen to the lecture. So listen, and then you practice again.
P3: I practice and the lecture will interrupt me again.
R: OK. Thank you. Has your lecturer ever used AI in the pronunciation classroom? If it
is no, have you used AI to practice or assess your English pronunciation?
P3: I think the lecturer never uses AI and for me myself I never use it to learn
pronunciation. because I usually use the online dictionary to learn it, such as Cambridge
or Expert Dictionary.
R: In your opinion, what are the strengths and limitations of feedback provided by AI
compared to feedback given by a human teacher? How do you prefer to receive feedback
and why? I'm going to say again, before doing this interview, we have already assessed
your pronunciation from the lecturer and AI and you receive the feedback and you know
your scores, what do you prefer?
P3: I think AI feedback has strength in consistency and objectivity, but I like human
feedback. And on the other hand, feedback from a human teacher can be more
personalized and empathy and a teacher can understand students' individual learning style
and provide feedback that is easy to learn linier with students' needs. And I prefer
receiving feedback from teacher because as I said before that it's gonna be easy. It's gonna
be easier for me to learn directly with the person and I can directly practice to correct
pronunciation with my lecturer.
R: Oh OK. Do you agree that AI can replace human teachers in giving feedback in an
EFL setting?
88
P3: In my opinion, I don't agree with that because some people believe that AI can’t be
just as effective as a human teacher in providing feedback because it can analyze a
language and also give feedback instantly and which can be helpful for students to
improve quickly. But on the other hand, a human teacher can modify and encourage
students in a way that a computer program cannot. Therefore, while AI can be a helpful
tool in providing feedback in an EFL setting, it may not be able to fully replace the role
of our human teacher. And both AI and human teacher have their strengths and limitations
and a combination of both may be the most effective approach in helping students learn
English as a foreign language.
R: OK, So what do you think about the strength and limitation between AI and teacher?
P3: AI, As I said before, AI feedback has trained consistently and objectivity, but mainly
like the personal touch of human feedback. And on the other hand, the human teacher,
they, can motivate and encourage us and also the teacher has limitations.
R: So you say that a teacher or lecturer can encourage you. So it means like can motivate
you?
P3: Yeah, motivation. Meanwhile, AI cannot.
R: OK, what is the limitation of the teacher that you think that you think it cannot be fine
in the teacher feedback but it can find in AI feedback?
P3: The limitation of teacher. To be honest, I have no idea for that because I prefer human
teacher feedback.
R: Oh, OK. Thank you so much. I want to ask you one question again. You already say
that you disagree that AI can replace teachers as giving feedback. In EFL setting, do you
think AI and teacher in the classroom might be collaborating and teaching or assessing
students in the pronunciation classroom?
P3: I think it might be. What is it? Yeah, collaboration between teacher and AI. And I
think it's going to be more effective like AI can give us the correct, correct pronunciation,
such as maybe the grammar or the pronunciation of etcetera in some way. And we can
also get the motivation and encouragement from the teacher, yeah, to be better.
89
R: OK. So, you agree that maybe eight of them can collaborate in the classroom?
P3: Yeah.
R: Thank you so much for your time
90
(R = Researcher) (P4 = Jupiter)
R: Hello, can you please explain your understanding of the role and importance of
feedback in the learning process?
P4: I think it's very important because the purpose of feedback is to improve the student
performance.
R: What's your understanding about feedback? It's...like... What's your understanding
about feedback because you just mentioned the importance of it, now what's your
understanding about feedback?
P4: According to my understanding, feedback is like after we have been tested we will be
corrected.
R: ok, so how important is feedback for your pronunciation?
P4: very important
R: why do you think it is very important?
P4: Because I can't judge myself, I need someone else. I need the expert.
R: okay so to improve your pronunciation you need the feedback from the lecturer. from
the expert. how do you believe feedback can affect the pronunciation of students,
especially you in learning a language? How important is it? How can feedback affect
pronunciation? So for example, after you get feedback, how influential is it in helping
you improve your English pronunciation?
P4: maybe for all students?
R: It's okay for you, it's your personal opinion about feedback.
P4: So, if we're given feedback, it's automatically like all the people who give the
feedback have the same pronouncements, one goal, no different.
91
R: So you mean like there is such an accent like British accent and American accent but
because you get feedback from the expert and the expert focuses on the American accent
so you will be more focused on the American accent right? Does feedback keep students,
especially you, engaged to keep improving your pronunciation? And does your lecturer
ever interrupt you when you are practicing speaking or reading aloud in class?
P4: For the second one, at that time in class, Miss X told us to read in the power point
presentation, like who wants to read. Then when a student reads, it's not interrupted, so
they read until the end and then the lecturer will correct it, like giving the correct
pronunciation word, giving the feedback at the end.
R: Then after you get feedback at the end, after you read, does that feedback makes you
feel engaged like motivated you to improve?
P4: It definitely motivates me more, like I automatically want to improve.
R: Don't you feel intimidated, don't feel like you're down?
P4: No
R: What kind of feedback can help you to improve your English pronunciation? Please
give reasons why?
P4: Feedback like, oral corrective feedback, eye to eye.
R: Oh that means you prefer face to face with the lecturer rather than in the class? Oh
okay, why?
P4: Personally, it is more focused and can feels the interaction, right, language is an
interaction so it's like deeper, more intense.
R: okay then we have already taken the pronunciation assessment last Monday and you
get the feedback from Miss X as a pronunciation lecturer, is that the kind of feedback you
prefer?
92
P4: Yes, it was, but at that time there were other students too.
R: oh yeah, okay so, you just want you and the lecturer?
P4: yep, if there is another student, I think it will give a little pressure for me.
R: oh okay, okay, thank you. Have you ever gotten feedback from your lecturer on your
pronunciation in class? If so, can you share your personal experience related to the
feedback?
P4: Yes, most of the time. We asked to read the sentences on the PPT.
R: Okay. What about from other lecturers?
P4: Other lecturers, no.
R: Oh, okay, thank you. What kind of feedback from lecturers do you think can improve
your pronunciation? Do you prefer formal or informal, written or spoken? Earlier you
said oral, spoken. So, do you prefer formal or informal?
P4: Informal. I feel more empathy from the lecturer, so it's more about personal
motivation.
R: Okay, has your pronunciation lecturer ever used AI in pronunciation class? If not, have
you ever used AI to train or assess your English pronunciation?
P4: So far, the lecturer has never used AI. But I have tried using the ELSA app. But I left
it immediately. I didn't feel like it.
R: Why?
P4: I get bored easily. Then it's too lonely, and too awkward
R: Because it’s like a robot, the robot is from your smartphone. What do you think are
the advantages and disadvantages of feedback provided by AI compared to feedback
provided by lecturers? Do you prefer to receive feedback from AI or lecturers and why?
93
Because earlier you mentioned that you have used the Elsa app to test or check your
pronunciation.
P4: For the advantages, maybe the explanation is more detailed, AI sometimes it's really
detailed, like until the smallest points are also noticed that we don't notice, and we can
use AI anytime anywhere. but the negative is sometimes hesitant too, because it's still in
the form of AI maybe, so we still doubt the results. we can't be sure, is this correct or not?
Because the rubric is not clear, the benchmark is not clear.
R: Okay, I agree. It's detailed and can be done anytime, not necessarily in class. Then the
weaknesses it's too lonely and I don't trust whether the score is correct or not. Is the
feedback, uh what, the pronunciation that you do is correct or not. As for the lecturer, the
lecturer herself, what are the advantages and disadvantages?
P4: Like, the lecturer is clear about her educational background. Like where she learned
from, it's more reliable.
R: What are the disadvantages?
P4: The disadvantage is that the lecturer is probably the opposite of AI, so the time is
already monotonous, the time is not as free as AI, you have to be in class.
R: And even in class, you don't get the opportunity to have face to face and or there are
your friends, you don't get the opportunity to pronounce the correct words, to do some
kind of assessment or test directly, maybe?
P4: Yes, if I want to get personal like eye to eye, I have to pay for it, if it's private with a
lecturer, for example, it's private as if you want, you have to pay more.
R: Alright, thank you. Do you agree that AI can replace human teachers in giving
feedback in an EFL setting? Do you agree with the statement that AI can replace the
position of a teacher or lecturer in an English class?
P4: For me, I don't agree.
R: Why?
94
P4: Because what was I said at the beginning is the importance of interaction, so if it's
with a robot, it's lonely, too rigid quietly. language is an interaction between humans
because humans are a social creature, so it's very different from AI.
R: But in your opinion, can the lecturer utilize AI technology in their pronunciation class,
especially for assessment and giving feedback?
R: collaboration is possible, because automatically the lecturer also monitors the results.
The results are like this, the lecturer knows, oh this is a bit something something.
R: And also do you agree if for example from collaborative, well about time constraint,
that is too much used by lecturer but with AI it can be more like not too much time used,
so it can be more effective in term of time do you agree?
P4: In terms of time, no. You automatically still have to check one by one. it doesn't
depend on AI. So we use AI to help, but we don't depend on AI.
R: Oh, okay. Then what you mean by helping, is it helping in terms of giving the
resources, because the AI that we just used, you already got feedback and assessment
from the AI and also with the lecturer yesterday, and in the AI that you used earlier there
was personalization, personalizing your focus area, which parts to improve. Is that how
collaborative it is, meaning like the AI is to help you train yourself and the lecturer is still
monitoring, is it like that?
P4: Yes, it can be, learning is continuous, so at first it was taught by the lecturer, then a
few meetings were handed over using AI, but still in the end it was tested again by the
lecturer.
R: okay, so it's just an alternative tool, but does that mean that the AI must be AI that has
been recommended by the lecturer?
P4: Yes, the one you already know
R: okay okay, thank you so much for your time for the interview.
95
(R = Researcher) (P5 = Saturn)
R: Okay, thank you, P5, for your time. Could you explain your understanding of the role
and importance of feedback in the learning process, especially in language learning?
P5: Okay, based on my understanding and personal experience, feedback is really
important to be given or proven, right? Feedback or criticism helps us grow into better
individuals. After receiving feedback, we will know where our mistakes are, and we can
work to improve them, making us better in the future.
R: Okay, how important is feedback for your pronunciation?
P5: I think feedback on pronunciation is quite important, very crucial even. It’s important
because when we pronounce words, we aim for native speakers to understand what we're
saying. So, feedback is really necessary to help us improve in pronouncing words in that
language.
R: Okay, thank you. In your opinion, how does feedback influence learners’
pronunciation, especially for you in learning a language?
P5: Feedback influences us by providing criticism or suggestions, so we can learn from
our mistakes and correct them. With continuous learning and improvement, we can
become better at pronouncing words in that language.
R: Okay, I agree. Can feedback encourage students to keep improving their
pronunciation, especially for you? And has a lecturer ever interrupted you during practice
speaking in class with the aim of correcting your pronunciation?
P5: Yes, feedback can definitely engage students to keep improving their pronunciation.
Because feedback helps us identify our weaknesses in certain areas, for instance,
consonants or vowels, and that can help us improve our pronunciation. It can also
motivate us to keep working on our pronunciation. As for lecturers, they never interrupt
unless I make a mistake in my pronunciation, but it’s not like they stop me. At the end of
the test, they tell me where my mistakes are and provide feedback.
R: So, you only receive feedback after you finish the entire sentence or paragraph?
96
P5: Yes, that’s right.
R: Okay. What kind of feedback helps you improve your English pronunciation? Please
provide your reasons!
P5: For me, effective feedback is direct feedback. Direct feedback really impacts me
because I understand more clearly where my mistakes and weaknesses are.
R: So, as we've discussed before, you have received feedback from Ms. X and from AI.
You prefer direct feedback like what Ms. X provided after you read everything and at the
end, you receive direct, verbal feedback, not written?
P5: Yes, exactly.
R: Okay, thank you. Have you ever received feedback from a lecturer regarding your
pronunciation in class? If yes, could you share your personal experience with that
feedback?
P5: Yes, I have received direct feedback from my speaking instructor. They said that my
pronunciation might sound correct and clear to native speakers if they listen casually, but
when they listened closely, they noticed I still had weaknesses in areas like "th" and "v."
R: How about the pronunciation lecturer in class?
P5: The pronunciation lecturer tends to focus more on the consonants we pronounce, like
in specific practices. For example, one day we might practice the sound "zeh," focusing
only on that part.
R: Oh, okay, so not an overall sentence? It’s more part-by-part? And does the
pronunciation lecturer often conduct practice sessions in class, like giving you sentences
or paragraphs to practice and then giving you direct feedback?
P5: Yes, often.
97
R: Okay, thank you. What kind of feedback from lecturers do you think could help you
improve your pronunciation? Do you prefer it to be formal or informal, and written or
spoken?
P5: Personally, I prefer informal and spoken feedback because I am a laid-back person.
If the feedback is delivered in a relaxed manner, I can follow along and make my own
improvements. But that doesn’t mean I can’t handle formal feedback, it’s just that I prefer
informal and spoken.
R: If it’s informal and spoken, would you prefer to receive it in front of your classmates
or one-on-one with the lecturer?
P5: Oh, it doesn’t matter. It can be in front of classmates too. That way, my friends can
also give their input.
R: So, if your lecturer were to give you feedback on your pronunciation test in front of
your classmates, it wouldn’t make you feel embarrassed or worried, right?
P5: Yes, no problem.
R: Has a lecturer ever used AI in your pronunciation class? If not, have you ever used AI
to train or assess your English pronunciation?
P5: The lecturer hasn’t used AI yet. They prefer to conduct direct practice, demonstrating
it themselves. As for me, I often use AI, like Google’s speech recognition tool called
"train your speech."
R: What do you think are the pros and cons of feedback provided by AI? Do you prefer
feedback from AI or from lecturers, and why? Because we’ve previously done two tests,
one with a lecturer and one with AI, both providing direct feedback: AI giving automatic
feedback, and the lecturer giving direct verbal feedback.
P5: In my opinion, one of the cons of AI feedback is that it can be affected by the
connection. For example, if we are in the middle of receiving feedback and it lags or
reloads, we might have to start the test over again. Otherwise, AI provides quite detailed
98
feedback. Personally, I prefer AI feedback, but I don’t have a problem with receiving it
directly from lecturers either.
R: Why?
P5: Because with AI, we can get recommendations on which consonants or vowels to
focus on, like word stress.
R: After receiving feedback from lecturers directly, what do you see as the pros and cons?
P5: The pros and cons of feedback from lecturers are that it may not be as detailed
because, after all, they are lecturers, and lecturers might have different opinions.
Sometimes this creates a conflict, like if Lecturer A says, "You did well here," and
Lecturer B gives different feedback. That difference in opinion can be a downside.
R: Okay, thank you. Do you agree that AI can replace human teachers in providing
feedback in an EFL (English as a Foreign Language) classroom context?
P5: Okay, I kind of agree and disagree. With the development of technology, we might
need AI for our growth, and it also reduces the need for human resources, which can be
more efficient. So, I agree.
R: If AI were combined with teachers in the classroom, would you prefer that, or do you
think AI could fully replace teachers?
P5: I would prefer a combination. There’s a teacher to guide us, and we can also get
feedback from AI and the teacher. That way, we can avoid conflicting opinions like
before. The teacher’s work is also made easier if the AI provides direct feedback, saving
class time, especially since there are many students, and it might not be possible to finish
everything in one day.
R: When it comes to motivation, do you feel more encouraged by your lecturer or by AI?
Because with AI, you’re alone in front of your device, while a lecturer can provide
motivation, empathy, and can assess based on your actual performance. So, which one
would you prefer?
99
P5: For motivation, I’d honestly prefer AI because it also gives suggestions on what to
focus on. But if I want to repeat myself to correct errors, I prefer a lecturer. That way, the
lecturer can directly assess my progress and provide feedback. With AI, I worry that if I
keep repeating something, there could be an error, like it might malfunction if I keep
repeating the same thing. I’m someone who likes to perfect things before moving on to
new material.
R: You mentioned errors earlier. Since AI can potentially make errors in assessment and
feedback, do you trust the feedback and scores from AI more or from your lecturer?
P5: If I had to choose, I trust feedback from lecturers more. But AI is more about
motivation, helping to build confidence. If AI says I'm doing well, it makes me think,
"Oh, so I should focus on this instead of that." But why do I trust lecturers' scores more?
Because lecturers are real; they listen directly and give feedback based on what they hear
firsthand. Considering AI's potential errors, like how it processes audio before analyzing,
I would place more trust in lecturers' feedback.
R: Speaking of lecturers' background in pronunciation, who have received education
specifically related to their subject, compared to AI’s assessment system, which do you
lean towards?
P5: Based on that, I would lean towards lecturers because they have more experience. AI
might still be new, like one or two years, while the lecturer might have 15 years of
experience in Australia. That’s why I prefer lecturers.
R: I want to ask you one more question. You mentioned earlier that AI could replace the
role of a teacher as a Human Assessor. Do you mean as a Human Assessor or for full
teaching?
P5: As a Human Assessor. So, just focusing on assessing and giving feedback. When it
comes to providing theory and other aspects, I prefer a lecturer.
R: Thank you so much, P5.
100
(R = Researcher) (P6 = Neptune)
R: Okay, thank you for your time. I’d like to ask a few questions. Could you explain your
understanding of the role and importance of feedback in the learning process?
P6: In my opinion, feedback plays a crucial role in the learning process because it allows
students to identify and understand where they may have made mistakes during their
learning journey.
R: How important is feedback for improving your pronunciation?
P6: Feedback is very important for my pronunciation. Pronunciation requires immediate
feedback, ideally from the instructor, to be really effective.
R: How do you think feedback can impact your pronunciation when learning a language?
P6: Feedback is helpful in guiding us in the right direction. In this modern age, we learn
languages from various sources, so feedback from teachers or lecturers is essential in
keeping us on the right path.
R: Does feedback encourage you to keep working on your pronunciation? Have
instructors ever interrupted you during activities like speaking exercises to correct your
pronunciation?
P6: That’s never happened to me, and hopefully, it won’t. Interrupting is generally seen
as impolite, even if the intention is good, as it can disrupt the learning process.
R: Does feedback encourage you to keep working on your pronunciation? For instance,
has feedback from an instructor motivated you to keep improving?
P6: Absolutely, feedback motivates me to work harder. Knowing where I went wrong
helps me improve my pronunciation.
R: What kind of feedback helps you improve pronunciation, and why?
P6: I prefer direct, practical feedback. This means pointing out where I went wrong and
showing me the correct way so I can practice.
101
R: Have you ever received feedback from an instructor on your pronunciation in class?
If so, can you share a personal experience?
P6: Yes, I have. I struggle with certain vowels and consonants, like connecting words or
omitting certain sounds. My instructor often points these issues out.
R: What kind of feedback from instructors is most effective in helping you improve your
pronunciation? Do you prefer formal or informal, written or oral feedback?
P6: I think direct, formal feedback works best, especially verbally. Written feedback
doesn’t show exactly how to position the mouth or tongue for proper pronunciation.
R: Previously, in Ms. X’s class, you read ten sentences before receiving feedback and
your score. Do you prefer this method or getting feedback after each sentence?
P6: I prefer reading all sentences first and then receiving feedback. Stopping for feedback
after each sentence can disrupt focus and make us overthink each sentence.
R: Has an instructor ever used AI in your pronunciation class? If not, have you ever used
AI to practice or assess your pronunciation?
P6: No, I haven’t, and neither have my instructors. I prefer learning directly from
instructors or observing native speakers for conventional learning.
R: What are the advantages and disadvantages of AI feedback versus instructor feedback?
Which do you prefer, and why? We’ve received feedback from both in previous
assessments.
P6: I prefer instructor feedback. Direct feedback on pronunciation from instructors is
more effective, as they can hear our voices clearly and observe mouth movements. AI
feedback depends on technology, which can be unreliable if, say, the device microphone
isn’t working properly. AI could be more effective if it provided visual aids or video
demonstrations, but I still think instructors offer the best method.
R: So, does it also depend on the instructor’s background, whereas with AI, you might
not know the full context?
102
P6: Yes, it also depends on the equipment’s quality.
R: So a disadvantage of AI is its reliance on technology. What do you think are its
advantages?
P6: The advantage of AI is that it’s based on aggregated data from native speakers
worldwide, while instructors may have accents or language influence from their native
languages.
R: So, with instructors, a benefit is their ability to understand a student’s specific
situation?
P6: Yes, instructors can assess issues like native language influence or accent differences,
which AI doesn’t account for as it maintains a consistent approach.
R: And what do you see as a disadvantage of instructor feedback?
P6: Different instructors may vary in pronunciation, which can create inconsistency. It’s
best for students to learn from more than one instructor.
R: Okay, thank you. Do you agree that AI could replace instructors in providing
pronunciation feedback, especially in EFL assessment?
P6: No, I disagree. AI is just a program, designed to reach perfection in specific areas,
but it lacks human qualities like empathy and understanding. We can assess each student’s
needs and emotions, but AI lacks that flexibility and insight.
R: What if AI and instructors were combined? Would you agree with that approach?
P6: I’d be open to a combination. We already have capable instructors, and AI could help
enhance our study when we practice at home.
R: Interesting. Thank you very much for your time.
