1. Introduction

1.1 Automated Writing Evaluation Tools 

As an important, yet difficult-to-acquire productive skill, writing can act as a means for learning other receptive and productive skills (Hyland & Hyland, 2006) and is considered an essential skill for students at the tertiary level (Narita, 2012). It makes thought available for reflection, encourages students to communicate, and, thus, brings about thinking and learning (Mekheimer, 2005). The sovereign position of writing is further elaborated on by Olshtain (2001), who argues "…the skill of writing enjoys special status–it is via writing that a person can communicate a variety of messages to close or distant known or unknown readers" (p. 207). 

Despite the importance of writing skill, Learners of English as a Foreign Language (EFL), international students studying in English-speaking countries, and their teachers experience multifarious unfavorable emotions (Tynan & Johns, 2015), and writing is among the most demanding competences they need to acquire (Afshari & Salehi, 2017). Besides, even native speakers of English exhibit difficulties with grammar and form in their writing in academic settings (Koroglu, 2014). Acknowledging the burdensome nature of writing, Nunan (1999, as cited in Afshari & Salehi, 2017) argues that "…producing a coherent, fluent, extended piece of writing is probably the most difficult thing there is to do in language" (p. 2).

Other serious challenges for writing teachers are to involve students in the writing process actively and to help them master grammar, punctuation, capitalization, and spelling, which are essential elements in delivering the message clearly and precisely (Perdana & Farida, 2019). Therefore, providing enough instruction and feedback on the said elements becomes highly important, especially in the initial stages of writing (Williams, 2004), when direct feedback is needed for improving accuracy (Dikli & Bleyle, 2014). This requires intervention and explicit input provided by teachers (Muller et al., 2017). However, teachers are unwilling to provide sufficient language support for all students (Murray, 2010), teachers’ feedback is not consistent, and the students do not understand them (Ranalli et al., 2017), and teachers might not find the time and have the patience to correct all the essays written by students and provide them with accurate and holistic feedback (Wilson & Czik, 2016). 

In order to help EFL students improve their abilities in writing, EFL teachers and researchers have adopted different initiatives and taken different steps such as offering corrective feedback, providing them with model essays, and employing computer software and online resources, technically called computer-assisted language learning (CALL) (Daniels & Leslie, 2013). Integrative CALL started in the last decade of the 20^th century, integrating technology more fully into language teaching and learning process through using authentic language in meaningful contexts (Rinaldo et al., 2011).

Automated writing evaluation (AWE) has been designed to help language learners with both input and correction on grammar, spelling, mechanics of language, and style among others. AWE tools have become increasingly ubiquitous and powerful (Daniels & Leslie, 2013). In its list of online AWE tools, Just Publishing Advice (2020) mentions 50 free writing software applications under seven categories of which one main category is that of automated feedback programs (AFPs). AFPs, including Grammarly, Ginger, WhiteSmoke, ProWritingAid, GrammarCheck, Hemingway Editor, Slick Write, InstaText, and Language Tool, mainly aim to help writers check, correct, and improve their writing components such as grammar, spelling, and mechanics of language.

1.2 Statement of the Problem

Previously, it was believed that success in technical translation projects depended only on two skills: command of the source language, which would allow the translator to comprehend the source material accurately, and proficiency in the target language, which would enable them to produce a text that clearly expressed the information contained in the source material. The importance of knowing the target language increased when the purpose of translation turned more toward communication. Herman (1993, as cited in Duběda & Obdržálková, 2021) noted that technical translation has the same stylistic objectives as technical writing: correctness, clarity, and concision. Byrne (2010) argues that technical translators would be able to produce translations that are up to par as long as they have the language abilities necessary to pass for the writing of a true expert in the subject matter. To put it another way, they must be very proficient writers in the target language, be able to write creatively in a range of text types, and have a knack for words in order to provide high-quality technical translations. Herman's maxim that one must first be a good technical writer in order to be a good technical translator has taken on new significance and dimensions as a result of various environmental changes that have an impact on the very nature of technical documentation and how it is produced and translated. The role of the technical translator has changed as a result of new research, increased collaboration at the professional and academic levels, new technologies and media for dissemination, as well as various legal considerations (see, for instance, Byrne, 2007; 2010). Technical translators these days not only translate but also write, create, and, more often, design and engineer texts (Byrne, 2010).

A translator translating into their mother tongue enjoys the perfect mastery of the target language with all its subsystems and rules and finds it simple to produce a quality target text because they draw on a natural linguistic repertoire that they have amassed from birth and that flows easily and naturally when needed. A translator translating into a foreign language, on the other hand, may not have complete command of the grammar and vocabulary of the target language, which inevitably leads to flouting target language norms and creating odd grammatical structures. Campbell (1998, p. 57) maintains that translators translating into the mother tongue avoid "the problem of lack of textual competence in the target language- in other words, native writers can manipulate all the devices that go to make up natural-looking texts.” Similarly, Newmark (1995, p. 180) emphasizes the importance of natural writing in translation, which can only be generated by the native speaker by arguing that a non-native translator "…will be caught every time, not by his grammar, which is probably suspiciously ‘better’ than an educated native’s, not by his vocabulary which may well be wider, but by his unacceptable collocation". Baker (2002) also considers the perfect mastery of the target language a decisive factor by asserting that translators should only translate into their language of habitual use.

No doubt, native English speakers are the best resources for translating technical Persian texts into English. However, this is not the case in the Iranian context, where a majority of technical texts, including academic papers, are translated from Persian to English by Iranian freelance translators whose native language is Persian. Therefore, AFPs could be considered a good resource to help them achieve the stylistic objectives of technical writing mentioned by Herman (1993, as cited in Duběda & Obdržálková, 2021), i.e., correctness, clarity, and concision.

1.3 Purpose of the Study

There is growing agreement among researchers on the benefits of using AFPs in EFL writing classes (Afshari & Salehi, 2017; Rahimi et al., 2020). As will be shown in the next section, most of the previous studies have evaluated more well-known software applications such as Grammarly, Ginger, iWrite, MI Write, PEG Writing, NC Write, Writing Roadmap, EssayCritic, and Summary Street in English writing classes. The present study, however, is intended to investigate the effectiveness of InstaText in improving the language quality of texts translated from Persian into English by freelance translators whose mother tongue is not English. In other words, the aim of the current study is twofold and as follows: 1. to see how the Persian-English freelance translators who used InstaText for editing their English translations of Persian academic papers perceive this online tool, and 2. to investigate the effects of the said tool on the language quality of the said translators' final translations through comparing the language quality of their translations before and after using InstaText.

2. Review of the Related Literature

The development of writing expertise entails sustained deliberate practice (Kellogg & Whiteford, 2009), which will help students both develop fluency and automaticity in lower-level writing skills (Kellogg, 2008) and have complete control of the central cognitive processes which are at work in composing (Graham et al., 2019). In addition to sustained deliberate practice, effective and frequent feedback also gets the lion’s share in the development of writing expertise (Hattie & Timperley, 2007), and such feedback can be beneficial to students, especially when it is provided immediately and in a localized, specific, and detailed manner by addressing both surface-level and content features of writing (Hattie & Timperley, 2007; Patchan et al., 2016). The problem, however, is that on the one hand students can hardly engage in sustained deliberate practice due to the limited time allocated to writing courses included in most curricula (Brindle et al., 2016). On the contrary, providing high-quality feedback in the manner described above is demanding both in terms of teachers' knowledge (Parr & Timperley, 2010) and their time (Dikli, 2010). Previous research (e.g., Matsumura et al., 2002) concluded teachers' feedback often focuses on low-level writing skills with little effect on students’ writing performance. Hence, improving students’ writing outcomes necessitates that alternative methods capable of helping enhance both students' sustained deliberate practice and teachers' feedback frequency and quality be identified.

Automated writing evaluation (AWE), defined as using all the computer technologies to
evaluate and score a written text, is one of such alternative methods. AWE tools were developed with three aims in mind: improving the practice-feedback circle (Kellogg et al., 2010); reducing teachers' evaluation demands (Palermo & Wilson, 2020); and creating fair scorings without any human imperfection (Stevenson & Phakiti, 2019; Wang et al., 2020). A central feature of any typical AWE tool is the provision of automated feedback aimed to help learners improve their writing quality by increasing students’ access to the cycles of practice and feedback (Palermo & Wilson, 2020). AWE tools use a scoring engine named automated essay scoring (AES) (Shermis & Burstein, 2003). AES, in turn, comprises a set of computerized methods for evaluating the text and providing a general numeric score; and it can be based on either latent semantic analysis (LSA) or natural language processing (NLP). LSA-based AES evaluates content, while NLP-based AES evaluates a broad array of aspects such as mechanics, usage, grammar, sentence variety, and the like (Nunes et al., 2021). In addition to the AES engine, AWE tools include a qualitative feedback engine providing feedback aimed at improving the quality of writing (Allen et al., 2016). 

Previous studies have shown that using AWE tools has some benefits for supporting writing skills acquisition. As regards students, it has been found that AWE tools help reduce the time spent on the writing task, increase the number of revisions students made, and improve their writing content (Franzke et al., 2005). In addition, it has been revealed through several studies that feedback provided by AWE tools has been beneficial to students in that they help them better both their writing attitudes and their opinions about their writing potential (Roscoe et al., 2018; Camacho et al., 2020), increase their autonomy and self-efficacy (Wilson & Czik, 2016; Wilson & Roscoe, 2020), enhance their writing quality, especially the mechanical aspects such as grammar and spelling, and accuracy due to the corrective feedback provided (Franzke et al., 2005; Kellogg et al., 2010; Wilson & Czik, 2016), improve their performance on state ELA exams (Wilson & Roscoe, 2020), and receive more accurate and objective scores (Warschauer & Grimes, 2008).

With respect to user perception, students using AWE tools tend to perceive them as valuable, since they incite the users to reflect on and become more aware of the writing process (Franzke et al., 2005). Furthermore, the feedback provided by AWE tools helps the users feel more confident and motivated, enjoy writing, rewriting, and revising more, and remain focused for longer (Tang & Rich, 2017; Palermo & Thomson, 2018).

On the negative side of the coin, some studies have pointed out that AWE tools are afflicted with several general disadvantages and limitations as follows: unclear feedback and judgments resulting from limited quantitative information on such issues as word repetition, word distribution, sentence length, etc. (Chou et al., 2016); limitedness to evaluating the content based on the specific prompts of the program and, thus, discriminating against students unfamiliar with technology (Khoii & Doroudian, 2013); and sufficing to common recommendations, which is a weakness in formative learning, profound meaning negotiation, and rich content development (Chen & Cheng, 2008). In addition, some studies show that students found some AWE tools' feedback too extensive and overwhelming (Ranalli, 2018) and that they cannot interpret the automated feedback without additional attention and support from the teachers (Palermo & Thomson, 2018; Wilson & Roscoe, 2020).

Although the findings of the previous studies show that the outcomes of using AWE tools are generally positive, such studies are mostly focused on investigating the effects of and/or users' perceptions toward AWE tools such as Grammarly (Koltovskaia, 2020), Ginger (Lastari, 2021), Criterion (Casal, 2016); iWrite (Qian et al., 2020), MI Write (Palermo & Wilson, 2020; Wilson et al., 2022) PEG Writing (Wilson & Czik, 2016; Wilson & Roscoe, 2020), NC Write (Palermo & Thomson, 2018), Writing Roadmap (Tang & Rich, 2017), and Summary Street (Franzke et al., 2005). In light of the foregoing, the present study can be novel in terms of its choice of the AWE, namely InstaText, which has not been studied to date to the best of our knowledge, and its participants' use of the software, i.e. for improving the quality of texts translated into English rather than originally written in English.

3. Method

3.1 Design

The present quantitative study was conducted in two phases. In the first phase, being of a one-group pretest-posttest design, the effect of using InstaText on improving the language quality of texts translated from Persian into English was investigated in a single group of participants. In the second phase, which was of a survey design and immediately followed the first, a perception questionnaire was administered to the participants to collect their impressions about using InstaText as an aid for improving the language quality of texts translated from Persian into English.

3.2 Participants

The participants of the study included 75 Persian-to-English translators who took a freelance translator admission exam following a call for freelance translators of academic papers by a reputable translation agency in Tehran, Iran. The participants' demographic information is presented in Table 1. The data for the first phase of the study were collected from the responses of all 75 participants to the questionnaire, but the data for the second phase were collected through the comparison of the translations submitted by 15 participants before and after using InstaText. 

Table 1

The Participants’ Demographic Information

3.3 Instrumentation

3.3.1 Persian-to-English Translation Pretest

Given the aims of the exam, the translation test was designed in such a way that it would cover the three main areas of Arts and Humanities, Engineering Sciences, and Life Sciences and represent the three challenging sections of academic papers, i.e., Abstract, Introduction, and Discussion. The reason for the latter decision was that as per our experience at the translation agency, our existing freelance staff's translations were afflicted with the largest number of issues in the said three sections. Considering the foregoing concerns, we designed a Persian-English translation test adding up to 502 words in total and comprising three paragraphs as follows: 1. the abstract of a paper in the area of life sciences (166 words); 2. the first paragraph of the introduction of a paper in the area of engineering sciences (138 words); and 3. the first paragraph of the discussion section of a paper in the area of arts and humanities (198 words).

3.3.2 Persian-to-English Translation Posttest

For the posttest, which followed the treatment immediately, the participants were required to use InstaText for editing their translations produced in the pretest phase. Adopting such a strategy, the researchers tried to minimize the typical threats to the internal validity of one-group pretest-posttest designs such as history, testing, maturation, instrumentation, regression to mean, and spontaneous remission (Campbell & Stanley, 2015).

3.3.3 InstaText

InstaText is an artificial intelligence-based online tool developed to help both native and non-native speakers of the English language with writing, editing, and revising academic papers, business proposals, marketing materials, and translations among others. Using artificial intelligence and language technologies, it generates recommendations and ideas on how to improve the text. For the purposes of the present study, we had the participants use the free trial version of InstaText.

The motivation behind choosing InstaText was that it claims to be a "personal writing assistant" (InstaText, 2021, Para. 3) aimed at simulating a native speaker (Para. 7) and capable of helping the user to produce high quality and efficient sentences and, thus, write clearly and accurately (Para. 5). It further claims that "one of the benefits that InstaText users so often praise is generating ideas" (Para. 5). In addition, InstaText claims to go much further than Grammarly by improving styling and word choice, correcting grammatical errors, and enriching the content to make it more readable and understandable (Para. 8). Furthermore, the researchers being practicing translators and translator trainers thought that InstaText could be very useful in the context of Iran's translation market, where there is high demand for translation of academic papers from Persian into English in a native-like manner, while the translators are native speakers of the source language rather than the target language.

3.3.4 Quality Assessment Rubric

In order to score the pretests and posttests, the researchers used a quality assessment rubric derived from LISA Quality Assurance Model 3.1, which LISA recommends for anyone needing an objective measure of translation and localization quality, including reviewers who want to evaluate translated texts (Martínez, 2014). The said rubric is comprised of several error categories, i.e., accuracy, terminology, language, country standards, formatting, and project manager instructions, and weighs errors as minor, medium, major, duplicate, and preferential.

However, out of the various error categories contained in the Quality Assessment Rubric, only quality issues under the Language Category were spotted and scored in this study. These issues include 1. Grammar, i.e., failure to follow target-language-specific rules of grammar, syntax, and punctuation; 2. Spelling, namely, misspellings, typographic errors, and incorrect accentuation and capitalization, and 3. Style, i.e., wrong register, inappropriate level of formality, not following style conventions, and unidiomatic usage of the target language.

3.3.5 USE Questionnaire: Usefulness, Satisfaction, and Ease of Use

Introduced by Lund (2001), the USE Questionnaire consists of three dimensions, namely, Usefulness (items 1-8), Satisfaction (items 24-30), and Ease of Use (items 9-23). The questionnaire is made up of 30 seven-point Likert Scale items, where point 1 stands for strongly disagree and point 7 stands for strongly agree. In addition, each item is provided with an NA option which means the item does not apply to a specific case. It must be noted that Ease of Use is further subdivided into Ease of Use (items 9-19) and Ease of Learning (items 20-23). Other than that, there are two open-ended questions, i.e., List the most noticeable weakness(es) and list the most significant strength(s). Last but not least, the questionnaire is available as an online form, which respondents can fill in and email directly.

Faria et al. (2016) maintain that the evaluation aspects presented in the USE Questionnaire are the most important factors to be used for evaluating software usability, and Lund (2001) states that the items are so easy-to-understand that the respondents can understand them with little training. The questionnaire has been used widely by several researchers (e.g., Faria et al., 2016; Salameh, 2017). Last but not least, the questionnaire has a public domain license, which means researchers do not have to pay for purchasing it. 

3.4 Data Collection and Analysis Procedure

A total number of 98 applicants had registered for the above-mentioned admission exam, which was administered online via Google Forms and comprised of different sections. However, the data from the following sections were collected for this study: 1. exam instructions; 2. applicant's details and demographic information; 3. a Persian-English translation test of academic papers (502 words and 100 minutes); 4. editing the Persian-English translation test of academic papers with InstaText (50 minutes); and 5. answering the USE Questionnaire (20 minutes). It must be noted that, in the interest of respecting research ethics, it was explained that sections 4 and 5 were optional and their data would be used for a research project. Further, to encourage and compensate the volunteers' time and efforts spent on these optional sections, one of the researchers held a free 4-hour workshop on the use of parallel corpora in translation.

The actual exam was held on Friday, January 15, 2021, from 9:00 A.M to 2:20 P.M. The applicants took the exam online at home. In the end, out of the 98 people who had initially registered for the exam, only 92 applicants took the exam. Upon reviewing the responses, we found that only 75 applicants out of the 92 had completed the optional sections 4 and 5, which provided the data for the present study.

Following the exam, the relevant sections of the exam which provided the required data for the present study were exported in CSV format. These included demographic information, the pretest, the posttest, and the USE Questionnaire. First of all, all of the participants were anonymized with P1, P2, P3, etc. codes, and their demographic information was extracted.

In order to extract the data for the one-group pretest-posttest phase, we needed to evaluate the preset and the posttest. Due to the fact evaluating 75 pretests and 75 posttests would prove too arduous a task, we decided to suffice to 15 samples. Choosing these samples was done using Research Randomizer, an online randomizing tool, which gave us the following numbers: P4, P10, P12, P23, P26, P40, P45, P46, P54, P58, P60, P62, P63, P66, and P68. Then the authors themselves evaluated the aforementioned participants' pretests and posttests in terms of the Language Category of the Quality Assessment Rubric. Subsequently, each participant's scores on the pretest and posttest were compared to determine to what extent InstaText might have aided that participant in improving their Persian-English translation in terms of grammar, spelling, and style. This was done using a t-test. 

In order to investigate the participants’ views about InstaText, each individual seven-point Likert scale item was analyzed separately by measuring the percentage for each scale. Then, the overall means for all four sections of the questionnaire are presented, followed by the total mean of the questionnaire. 

As regards the second phase of the study, the scores of all 15 participants in both the pretest and posttest and their mean differences are presented, as well as their scores on the subsections of style, grammar, and spelling. Following these, the data were checked for normality through the Shapiro-Wilk test. Finally, paired-sample t-tests and Wilcoxon Signed Ranks test were employed to find out if InstaText had influenced the participants’ performance in the grammar section, style section, spelling section, and total test.  

4. Results

4.1 Users’ Questionnaire Results

As mentioned before, to examine the attitudes of the participants toward InstaText, the responses of 75 participants who used this online tool during the course were recorded. The questionnaire consisted of 4 parts, namely, usefulness (1-8), ease of use (9-19), ease of learning, (20-23), and satisfaction (24-30). The results of each section are presented separately. First of all, the descriptive statistics of the Usefulness section, comprising 8 items, are presented in Table 2.

Table 2

Descriptive Statistics of Usefulness 

Since the number of points on the scale ranged from 1 (strongly disagree) to 7 (strongly agree), it can be seen that all seven items have been ranked more than average. This shows that the participants believed the software was really useful (5.84, the highest mean) and could help them be more productive and effective in their translation, save their time, reach their goals; and, above all, it met all their needs. The highest mean belonged to item number 3, which is at the heart of this section. However, the lowest mean was related to item number 8, stating that it could do everything I expected, which is quite acceptable since every online tool has some limitations and problems. All in all, the mean for this section was 5.37 out of 7, which shows the participants were happy with this aspect of the software. 

The next category in the questionnaire was Ease of Use, consisting of 11 items. This aspect, having the largest number of items, can be considered one of the very important aspects of every language learning tool. The descriptive results of this section are shown in Table 3 below.

Table 3

Descriptive Statistics of Ease of Use    

As the results clearly show, the mean values of all the items are more than 5, with the lowest mean belonging to item 16 at 5.24, which is not low in its turn. This item is related to the inconsistencies while using InstaText. However, the results show that it is both easy to use, simple to use, and user-friendly (6.00, 6.08, and 5.98 respectively). Items 12, 13, 14, 15, 18, and 19 can be considered middle-ranked items in this category, with their means ranging from 5.43 to 5.68. The total mean for this category is 5.63, which is the second-highest mean among the four categories. 

The category Ease of Use is closely linked with another category called Ease of Learning, forming the Ease aspect of the questionnaire. The category Ease of Learning had the lowest number of items (only 4 items), but it enjoyed the highest total mean among the 4 categories (6.03). 

Table 4 

Descriptive Statistics of Ease of Learning

As the statistics in Table 4 show, the participants did not experience any difficulty in learning how to use InstaText, became skillful with it, and could easily remember how to use it. Thus, it is safe to say that besides ease of use, the application was easy to learn too. These two aspects can make it one of the best options for writing and translation teachers while teaching. The last seven items in the questionnaire belonged to the Satisfaction category. Table 5 presents the data related to this category and the total mean of the questionnaire. 

Table 5

Descriptive Statistics of Satisfaction

The analysis of the responses revealed that the participants were highly satisfied with InstaText. Although this category was ranked third among the four categories, the total mean of the category was quite high (5.60). The participants were satisfied with the application (5.74), felt the need to have it (5.60), and were even ready to recommend it to their friends (5.81). The last point related to the questionnaire is that the total mean of the items was 5.61, which shows that this application was easy to use and easy to learn, proved useful, and was regarded as satisfactory. 

4.2 Pretest-Posttest Results

As mentioned before, the participants translated a text from Persian to English (pretest) and then used InstaText for editing the same text (posttest). In order to see to what extent InstaText had been effective in improving the quality of the participants’ second text, the participants’ scores in the pretest (including grammar, spelling, and style) were compared with their scores in the posttest with the same sections. Besides, the mean difference for each of these sections is shown in Table 6. All these details are shown for each of the fifteen participants separately. It must be noted that due to the nature of the AFPs, in general, and InstaText, in particular, the users are provided with corrective suggestions that they can choose to accept or dismiss. Therefore, the results having been reported here do not indicate which suggestions they have accepted and which ones they have dismissed. Our judgments of the efficacy of InstaText for improving the three sections are based on the scores which the participants obtained on the pretest and the post-test. 

Table 6

Participants’ Scores on the Three Sections of the Pretest and Posttest

The results show that the scores of most of the participants, 10 out of 15, in the grammar section improved. The amount of mean difference ranged from 7 to 31. One of the participants, P60, did not have any improvement in his writing score. On the downside, three participants had a lower mark in their grammar posttest, with their scores decreasing by 1, 19, and 30 scores respectively. Thus, it seems that InstaText could help most participants write more accurate sentences in the posttest. With regard to spelling, one can clearly see that more than 50% of the participants did not experience much change in their spelling scores, while the scores of four participants decreased. Only three participants, P40, P46, and P66, had better scores in their spelling posttest. Finally, it was revealed that the biggest change was related to the scores of participants in the style section of the test. Twelve participants out of fifteen took advantage of InstaText in improving their style, with mean differences ranging from 4 to 11. One participant, P40, did not change at all in his style score, while another participant, P23, used InstaText to deteriorate the style of his writing. 

In order to find out if using InstaText helped participants improve the quality of their writing in general and in three specific areas of grammar, spelling, and style; the researchers compared the results of the pretest and posttest. Before doing so, the data were checked for normality. Using the Shapiro-Wilk test which is suitable for samples fewer than 50, it was found that pretest scores, posttest scores, grammar pretest, grammar posttest, style pretest, and style posttest were all normal with significance values of .16, .61, .09, .25, .41, and .07 respectively. However, the spelling pretest and spelling posttest scores were not normally distributed. Thus, a series of paired-sample t-tests were run to find out if using InstaText had been effective in the aspects displaying normal data. The results of these t-tests are presented in Table 7.

Table 7

Paired-sample T-test Results for Grammar Section, Style Section, and Totals Score 

As the results in Table 7 show, a paired-sample t-test was run to compare the overall quality of the texts written by participants before and after using InstaText. On average, the learners performed worse before (M = 66.75, SD = 11.30) than after using InstaText (M = 71.15, SD = 9.13). This improvement, 4.40, was statistically significant, t(14) = -2.96, p = 01. Therefore, it was concluded that InstaText could significantly change the scores of participants from the pretest to the posttest. Considering the fact that the spelling section, which accounts for one-third of the performance, did not change much as a result of treatment, this improvement must be regarded as really significant. 

As regards the grammar section, the results from the pre-test (M = 54.13, SD = 26.91) and post-test (M = 60.13, SD = 21.37) indicated that using InstaText did not result in a significant improvement in the accuracy of the texts, t(14) = -1.41, p = .17. Finally with regard to style, the results from the pre-test (M = 50.33, SD = 7.18) and post-test (M = 55.20, SD = 7.54) indicated that using InstaText led to a significant improvement in the style of the texts, t(14) = -2.59, p = .02. Thus, it can be concluded that using InstaText helped participants improve the overall quality of their texts and their style, but it could not help them write more accurate sentences.

The last point about the effect of using InstaText is related to the spelling section. Since the data of the spelling section was not normally distributed, a Wilcoxon signed-rank test was run to check for the possible difference between the spelling pretest and posttest. The results of the Wilcoxon signed-rank test showed that using the said tool did not elicit a statistically significant change in the spelling scores of the learners (Z = -.689, p = 0.491). 

5. Discussion

The current study was conducted with the purpose of investigating both the effect of InstaText on improving language quality and users' perceptions toward it. The results revealed that the participants perceived the tool as really useful in that it helped them be more productive and effective in their translation, save their time, and reach their goals. Besides, it was found that InstaText was not only easy to use but also easy to learn since the participants did not experience any difficulty in learning how to use it. Added to that, the participants were so satisfied with the tool that they said they would highly recommend it to their colleagues. These findings, which are in line with those of most previous studies (e.g., Franzke et al., 2005; Graham et al., 2015; Tang & Rich, 2017; Palermo & Thomson, 2018), may be attributed to three factors: 1. InstaText is an easily accessible cloud-based tool; 2. both its editing process and the relevant toolkit are very intuitive; and 3. its editing environment is quite similar to that of Microsoft Word, meaning that translators, being already adept at using Microsoft Word, can easily migrate to. In addition, the participants' satisfaction with the tool lends support to InstaText's claim that it goes much further than Grammarly by improving style and word choice, correcting grammatical errors, and enriching the content to make it more readable and understandable (InstaText, 2021, Paras. 5,8). It must be noted that the forgoing findings are in contrast to those of Palermo and Thomson (2018), Ranalli (2018), and Wilson and Roscoe (2020).

In the pretest-posttest phase of the study, we found that InstaText did not help the participants make significant progress in grammar and spelling, which contradicts the findings of most previous studies (e.g., Franzke et al., 2005; Kellogg et al., 2010; Wilson & Czik, 2016; Wilson, 2017). In fact, InstaText did help improve 10 participants' scores tangibly (from +7 to +31); still, such improvement was traded off for the decrease in 4 participants' scores (-1 to -30). Further, such a finding can be attributed to the fact that the participants were freelance translators, which means they were already good at grammar. The reason why we could not observe any significant improvement in spelling is that the participants typed their translations in Google Forms, which suggests corrections for misspelled words. This is supported by 8 participants' full spelling scores in their pretests, meaning that no spelling improvement was needed. Having a closer look at Table 6, we can see for 3 participants, InstaText improved their spelling scores noticeably (from +9 to +19). As regards the remaining 4 participants whose spelling scores were reduced slightly as a result of using InstaText, the reduced scores were related to extra blank spaces, included in our TQA Rubric but not amended by InstaText. On the other hand, the participants made remarkable progress in terms of style in the wake of editing their translations with InstaText, which not only supports Wilson and Czik (2016), Palermo and Thomson (2018), and Wilson and Roscoe (2020) but also corroborates InstaText's claim that it is capable of helping the user produce clear and efficient sentences with improved style and word choice (InstaText, 2021) by suggesting lots of amendments to the user to accept or reject. Overall, using the tool resulted in remarkably higher scores in the posttest as compared to the pretest, which is consistent with Graham (2006), Graham and Perin (2007) Graham et al. (2012), and Wilson and Roscoe (2020). Such significant improvement in overall scores could be because AWE tools prove more fruitful with more proficient users (Koltovskaia, 2020) and because they help better users' writing attitudes (Roscoe et al., 2018; Camacho et al., 2020).

6. Conclusion

The present study showed that the participants perceived InstaText as a useful, easy-to-use, and satisfactory assistant for editing their Persian-to-English translations. This was also confirmed by the positive effect of the tool on improving language quality in the pretest-posttest phase of the study. Therefore, InstaText can be regarded as an AFP capable of helping both students and freelance translators to exert metacognitive control over the central cognitive processes involved in writing through the provision of effective and frequent feedback immediately and in a localized, specific, and detailed manner.

The findings of this study have to be seen in the light of some limitations as follows. First, we conducted the study using the one-group pretest-posttest design, not being free from threats to the internal and external validity; hence, it is suggested that future studies on InstaText use the true experimental design. Second, the participants of the present study were selected using volunteer convenience sampling, so the study can be replicated using more systematic sampling methods. Third, we studied InstaText from the perspective of freelance Persian-English translators using a questionnaire. Hence, we recommend that the tool be evaluated from students, teachers, and professional writers' viewpoints by using in-depth interviews.

The findings of the study are expected to contribute to our knowledge by signifying translators' attitudes toward AFPs and helping discover the strengths and weaknesses of InstaText. The study can also enable researchers as well as teachers to have a better picture of how AFPs, in general, and InstaText, in particular, could contribute to improving writing quality and ability. With such knowledge, researchers, curriculum designers, and teachers would be able to develop specific, appropriate, and creative pedagogical methods for making effective use of online AFPs.

Acknowledgments: The researchers are really thankful to all the participants who stayed with us until the end of the research and helped us with data collection.