Abstract

English:

English:

This article makes a contribution to translation quality assessment in the functionalist paradigm. It presents a target-group-centered empirical method for determining the comprehensibility (and thus an important aspect of the skopos adequacy) of texts: optimizing reverbalization using thinking aloud and log files.

Keywords: translation quality assessment, comprehensibility research, skopos adequacy, valutazione della qualità della traduzione, studi sulla comprensibilità, adeguatezza rispetto allo skopos

1 Assessing translation quality within the functionalist paradigm[1]

Both translation theorists and translation practitioners agree that measuring the translation quality of pragmatic texts, such as operating instructions, popular science texts, and commercial correspondence, involves determining the degree to which they are skopos-adequate (cf., e.g., Reiß / Vermeer 1984; Holz-Mänttäri 1984). The question that has not been answered yet, however, is how skopos adequacy can be determined empirically without leaving the functionalist paradigm (House 1997:12, 14). According to Hönig (1998:49), translation quality assessment within the functionalist paradigm is not possible without recourse to methods of readability and comprehensibility research. He criticizes that, when it comes to translation quality assessment, even within the functionalist paradigm, “linguistic evaluation on a contrastive basis [i. e., on a source-text-oriented basis] is re-integrated through the backdoor with the critic claiming that it is a typical reader’s response” (Hönig 1998:15). He goes on:

I put great hope in the latest developments in the field of readability studies and text optimisation. [...] It’s not easy to test whether one text is more readable than another one. Perhaps the fairly young discipline may eventually provide an empirical basis for what we are looking for: how can we establish, empirically, that certain texts function better than others. Such an approach would give us some kind of basis for evaluation. (Hönig 1998:49)

This paper will focus on a method to fill this gap.

Reliable results on the skopos adequacy of instructive texts can be obtained by usability testing (cf. Rubin 1994). The reason for this is that instructive texts make the reader do things. These actions are exteriorizations of how the reader has comprehended the text and make his or her comprehension observable for others. The skopos adequacy and - as an essential component of it - the comprehensibility of non-instructive texts, on which I will concentrate in what follows, however, is harder to analyze.

To determine the comprehensibility of both instructive and non-instructive texts numerous methods have been devised. Schriver (1989) provides a review of these methods and classifies them into three categories: “text-focused methods”, “expert-judgement-focused methods” and “reader-focused methods”.

An example of text-focused methods is the application of readability formulas. These are popular because they can easily and quickly be applied by means of computer programs but do not give us deeper insight into the comprehensibility of the texts to which they are applied because they take into account only certain lexical, syntactical, and stylistic aspects of what makes a text comprehensible or incomprehensible.

An expert-judgement-focused method is the employment of my Karlsruhe comprehensibility concept (Göpferich 2001; 2002), which represents an extended and improved version of the comprehensibility concepts by the Hamburg group of psychologists Langer, Schulz von Thun & Tausch (1993) and by Groeben (1982). It has proved a reliable instrument in pre-optimizing non-instructive texts (cf. Göpferich 2006b:154ff.) but cannot replace target-group-centered empirical research into text comprehensibility (cf. Schriver 1989: 247: “But expert-judgement-focused methods should not be used in isolation; they need to be supplemented with other document evaluation methods, particularly those which are reader-focused”.

Reader-focused or, to be more precise, target-group-focused methods undoubtedly provide the least speculative and most reliable results on text comprehensibility because this is a relative text quality which depends on the audience, whose comprehension and comprehension problems are central for its evaluation. Schriver (1989: 241) emphasizes that an optimal text evaluation method should provide two types of information: “(1) information about whole-text or global aspects of text quality, and (2) information about how the audience may respond to the text”. The method presented in this article belongs to the category of target-group-focused methods.

Other types of target-group-focused empirical methods employed so far in comprehensibility research are cloze procedures, questions on the texts whose comprehensibility is to be determined, and reproductions of such texts. These methods have the disadvantage, however, that they measure either only aspects of the texts’ comprehensibility (e. g., the predictability of words and phrases that fill gaps, the comprehensibility of words or passages relevant for answering the questions asked) or merely their rough overall comprehensibility. Furthermore, some of these methods lead to a confusion of the concepts of comprehensibility and retainability (cf. the research review in Schriver 1989:244ff. and in Göpferich 2006b:Chapter 4).

In this article, I will present a target-group-centered empirical method for determining the comprehensibility (and thus an important aspect of the skopos adequacy) of texts. It takes into account every detail of the texts to be analyzed, is independent of the texts’ retainability, and does not only allow the researcher to detect where the texts are incomprehensible, but also where they are hard to understand and where they give rise to misunderstandings or a demand for further information which is not given in the texts. I call this method optimizing reverbalization using thinking aloud and log files.

Since the publication of Ericsson’s and Simon’s book Protocol Analysis: Verbal Reports as Data in 1984 (Ericsson/Simon 1999), thinking aloud has found its way into the exploration of writing processes (cf. recently Schindler 2004) and (interlingual) translation processes (cf. the special edition of Meta edited by Lee-Jahnke 2005). To gain deeper insight into these processes, the method of thinking aloud has been combined with the use of the software TRANSLOG developed by Jakobsen and Schou (Jakobsen 1999). This software records (logs) all keystrokes and mouse clicks during writing processes as well as the time intervals between them without the user of this software realizing this.

I have adopted the thinking-aloud method combined with the use of TRANSLOG to investigate the comprehensibility of a popular science text on diabetes. The text is not a translation, but an original; the method of optimizing reverbalization with thinking-aloud and log files, however, can be used to determine the comprehensibility of any pragmatic text within the functionalist paradigm, no matter whether it is a translation or an original text.

The application of thinking-aloud in comprehensibility research is not new either. In the past, however, this method was only used either in reading comprehension research to determine text comprehension during reading processes, or it was used during revision processes (for a review of such studies cf. Schriver 1989:249). As translators know from experience, however, text reception is much deeper when a text is not just read for reasons of interest, but in order to translate it into another language or to paraphrase it (cf. also Shreve et al. 1993). Working with texts in this way often makes obstacles to comprehension arise which the reader would never have realized without the additional text processing task. For this reason, in the experiments described in the following reading a text was combined with its optimization. This optimization can be considered a type of intralingual translation. Thinking-aloud during revision processes has the disadvantage that this is no reader-focused, but an expert-judgement-focused method and the subjects have to speculate about the text’s comprehensibility for its audience.

In my article, I will give a detailed description of the method I employed and present what it reveals about the comprehensibility of a diabetes text. Subsequently, I will evaluate the experimental design and make suggestions on how it can be improved in future research.

2 The methods employed

Five subjects, who belong to the target group of the popular-science text on diabetes reproduced in [url=http://www.intralinea.it/parallel/eng_more.php?id=810_0_31_0_M%]Appendix A[/url], were asked to reverbalize this text in TRANSLOG in such a way that the result was optimally comprehensible for its target group. The target group is specified as follows on the website where the text to be optimized appears: “These contributions provide basic information on diabetes mellitus for which no prior knowledge is required.” (Deutsches Diabetes-Zentrum 2004; my translation). During the experiment, the subjects had to think aloud (“level 1 verbalizations” according to Ericsson and Simon 1999: 79).

2.1 The subjects

All subjects were female and either students in the degree programme “Translation and Interpreting” at the Department of Translation Studies of the University of Graz, had graduated from this programme, or were lecturers there. Their mother tongue was German; four of them were Austrians and one (YG) came from Switzerland. A short description of the subjects’ educational and professional background is given in [url=http://www.intralinea.it/parallel/eng_more.php?id=807_0_31_0_M%]Table 1[/url]. [url=http://www.intralinea.it/parallel/eng_more.php?id=808_0_31_0_M%]Table 2[/url] provides information on their physical and psychological condition during the experiment as described by themselves.

Although all subjects belong to the target group of the text to be analyzed, they are not representative of this target group. As can be seen from [url=http://www.intralinea.it/parallel/eng_more.php?id=807_0_31_0_M%]Table 1[/url], their education and training is certainly above that of the average reader. This means that whatever is incomprehensible for them can also be regarded as incomprehensible for the intended readership in general. Since the subjects were asked to optimize the text for the intended readership (and not only for themselves), they may have optimized sections of the text which they have found comprehensible for themselves, but considered incomprehensible for people with lower education. Their judgement on such elements of the text must be considered to be speculative. To reduce subjectivity, I have taken into account how many of the five subjects judged elements of the text to be optimized incomprehensible or hard to understand. We must not forget, however, that people with a lower educational standard might find additional things incomprehensible that were not criticized or optimized in the experiment. Furthermore, they may not realize that they have not understood certain aspects.

Selecting subjects with higher education and with at least some experience in translation has the advantage that these subjects have a higher metacognitive competence[2] with regard to the evaluation of their text comprehension (cf. Baker 1985) and apply a wider range of evaluation criteria (Baker 1989:15; cf. also Section 4). Furthermore their meta-linguistic and meta-communicative competence allows them to give a more detailed and precise description of their comprehension problems and optimizing maxims and strategies than persons with no education and training in this field. As a result, their thinking-aloud protocols are more illuminating.[3]

Although what these subjects’ thinking-aloud protocols reveal about text comprehension may not be representative of the entire target group, it can at least provide questions, which can be used to find out how well subjects with lower educational standards have understood certain passages of the text. Asking such questions is important because empirical studies have shown that subjects with lower education (such as children) do not become aware of certain comprehension problems unless they are explicitly asked for them (cf. the research review by Baker 1985).

2.2 The assignment

Each subject had to reverbalize the text in [url=http://www.intralinea.it/parallel/eng_more.php?id=810_0_31_0_M%]Appendix A[/url] in such a way that it would be tailored to the requirements of its intended readership. Passages which the subjects considered perfect could be copied into the target version. Prior to the actual experiment a trial run with a different text was carried out to acquaint the subjects with the functionality of TRANSLOG (editing functions and TRANSLOG dictionary). Only after all the questions on the software and the test setting had been answered was the actual experiment begun. The assignment was explained to the subjects by the supervisor and also handed out to them in writing. The dictionary entries provided with the source text in TRANSLOG are reproduced in [url=http://www.intralinea.it/parallel/eng_more.php?id=812_0_31_0_M%]Appendix C[/url]. No other material could be used during the experiment.

During the experiment, each subject was sitting in a quiet room[4] together with a supervisor. The subjects wore head sets; their verbalizations were recorded with the freeware AUDACITY and exported in MP3 format. The recordings were transcribed according to the GAT conventions for basic transcripts (Selting et al. 1998) and then proof-read by at least one other person. The complete transcripts of all five subjects can be downloaded as a PDF file from Göpferich (2005). During the experiment the subjects were not put under time pressure (cf. [url=http://www.intralinea.it/parallel/eng_more.php?id=808_0_31_0_M%]Table 2[/url]). They were informed that what was analyzed in the experiment was not their competence but the comprehensibility of the diabetes text. After the optimization process, the subjects were asked whether they had any questions for a diabetes specialist which had cropped up during the experiment and which they could not answer using the information in the text and the TRANSLOG dictionary. These retrospective interviews were recorded and transcribed too. They can be found at the end of each of the transcripts in Göpferich (2005).

2.3 The data

Apart from the information given in [url=http://www.intralinea.it/parallel/eng_more.php?id=807_0_31_0_M%]Table 1[/url] and [url=http://www.intralinea.it/parallel/eng_more.php?id=808_0_31_0_M%]Table 2[/url], the experiment provided the following data:

1. the optimized versions of the diabetes text (the version by YG is given in [url=http://www.intralinea.it/parallel/eng_more.php?id=813_0_31_0_M%]Appendix D[/url] as an example),

2. the log files (an extract from the log file of YG is given in [url=http://www.intralinea.it/parallel/eng_more.php?id=814_0_31_0_M%]Appendix E[/url] as an example), and

3. the thinking-aloud protocols (TAPs) as well as the protocols of the retrospective interviews (RIPs; cf. Göpferich 2005).

2.4 Data analysis

The data were analyzed as follows:

1. Each subject’s version was compared to the original text. Passages in which changes had been made were numbered and juxtaposed to their original version in a table. For each change the comprehensibility dimension in which this change occurred according to the Karlsruhe comprehensibility concept[5] (Göpferich 2001; 2002) was determined as well as whether the change really improved the text (+), made it worse (-), or represented neither an improvement nor a deterioration (*). [url=http://www.intralinea.it/parallel/eng_more.php?id=815_0_31_0_M%]Table 4[/url] shows these types of information for subject YG. As can be seen from this table, there are no modifications which cannot be attributed to one of the comprehensibility dimensions of the Karlsruhe concept.

2. The TAPs were analyzed for comments on why the changes had been made as well as on the maxims the subjects had and the strategies they used in optimizing the text.

3. The TAPs were analyzed for further comments on the quality of the original text which did not result in changes in their optimized versions.

4. The questions which the subjects had on the text, which they could not answer using the material available to them (i. e., the text to be optimized itself and the TRANSLOG dictionary), and which they therefore would have liked to ask a specialist were collected. If we start from the assumption that an optimally comprehensible text does not give rise to questions in the reader’s mind which it does not answer, these remaining questions are additional reliable indicators of deficiencies with regard to comprehensibility. This also applies to dictionary look-ups.

5. For each element of the original text that had been subject to criticism or questions in the experiment, I determined how many subjects had criticized it or had questions on it. The more subjects commented on it, the higher the probability that it may really lead to comprehension problems.

6. An optimized version was written in which the criticism of all subjects was taken into account and which answers the questions they had. In this optimized version, only real improvements suggested by the subjects (cf. “+” in column 5 in [url=http://www.intralinea.it/parallel/eng_more.php?id=815_0_31_0_M%]Table 4[/url]) were considered. If a subject formulated a maxim for a specific section of the text without providing a solution fulfilling this maxim, I tried to provide such a solution by myself. Deteriorations (-) and ‘cosmetic’ (*) changes were ignored. Linguistic mistakes made in the source text or by the subjects were corrected in the optimized version.

7. The subjects’ maxims and strategies were analyzed and classified. For each subject the repertoire of maxims she had and strategies she used was determined. From these results, conclusions for text production didactics can be drawn. The subjects’ maxims and strategies, however, are beyond the scope of this paper. I will mention some of them here, a detailed analysis of them will be provided in Göpferich (2006a).

3 Results

In the following, I will give a survey of all the elements of the original text that either one or more subjects in the experiment considered hard to understand or incomprehensible as well as of the questions the text gave rise to in the subjects’ mind without providing an answer. For each element, extracts from the TAPs and/or RIPs will be quoted which show that the subject(s) found it difficult and, if applicable, what maxims and strategies they followed to improve the corresponding passage. A distinction is made between a) completely incomprehensible elements of the source text and passages giving rise to questions that are not answered in the text and b) passages which are simply hard to understand. A complete list of all the items criticized by the subjects is given in [url=http://www.intralinea.it/parallel/eng_more.php?id=809_0_31_0_M%]Table 3[/url] (Section 4).

EK is the only subject in my experiment who, in her optimization, also took into account that the text to be optimized is an online text which allows hyperlinks. In the following, I will only consider those comments by her which refer to the text’s comprehensibility in general, and not those referring to the question whether it fulfills the requirements of the online medium (this will also be subject of Göpferich 2006a).

3.1 Incomprehensible elements and missing information

1. The text says that a distinction is made between two types of diabetes, but then three  types are introduced: type 1 diabetes, type 2 diabetes, and gestational diabetes. This makes YG wonder whether gestational diabetes is a variant of type 2 diabetes (cf. TAP YG 29-30) or a type of its own. Thinking aloud she says:

wenn i jetzt wüsst, (-) ob (.) der (.) schwangerschaftsdiabetes, (.) ob de::s jetzt (.) typ drei isch <<tippt öfters auf die Tastatur ohne zu schreiben>> odersch, (—) typ=zwei (-) <<sich selbst beim Tippen diktierend> werden generell, im allgemeinen> (-) woteva (—) <<sich selbst beim Tippen diktierend> im allgemeinen zwei typen unterschieden> (TAP YG 82-88).

She solves the problem by adding “im Allgemeinen” (In general, a distinction between two types of diabetes is made.) and referring to gestational diabetes as a “Spezialfall”, a special type of diabetes. The log file (cf. [url=http://www.intralinea.it/parallel/eng_more.php?id=814_0_31_0_M%]Appendix E[/url]) reveals that she first puts down “zwei Typen” (two types of diabetes), then changes this into three types, and uses “zwei Typen” again in her final version. In the retrospective interview (RIP YG 476-482), she says that she would like to ask a specialist about this because she is still not sure which one is the correct version.

EK also has the problem of not knowing whether gestational diabetes is a variant of type 1 or type 2 diabetes or a type of its own. She thinks about this three times during the optimization process, which shows that this is a serious problem for her (cf. TAP EK 12-18, 326-329, 461-465). In the end, she chooses a similar solution as YG.

JS’s TAP reveals that she has the same problem as YG and also chooses a similar solution:

<<sich selbst beim Tippen diktierend> oder alterszucker> (4.0) ja da sollt ma aber fast einen einleitenden satz schreiben, .hh ah:: (-) weil dann ist das a bissl (.) klarer vielleicht (.) .h <<sich selbst beim Tippen diktierend> grundsätzlich lassen sich (-) .h sich zwei arten von diabetes> ja weil DES klingt a bissl <<flüsternd> komisch> (TAP JS 332-337)

ah so da gibts noch eine extraart von diabetes; schwangerschaftsdiabetes. (3.0) <<pp> ah:.> ah so (.) dann passt das vielleicht eh mit dem grundsätzlich, (TAP JS 355-358)

Asked for the classification of the types of diabetes that is made in the text, SF, too, comments that she finds it illogical that gestational diabetes is not counted as a third type (cf. RIP SF 567-590).

To my mind, the real problem in the text here is that the author informs us about what happens in the bodies of patients with type 1 diabetes and type 2 diabetes, but not about what happens in the bodies of women with gestational diabetes. If this information were given, it would be clear that gestational diabetes is neither a variant of type 1 nor of type 2. In contrast to type 1 diabetes, which occurs when the body produces too little or no insulin, and type 2 diabetes, which occurs when the body cannot use the insulin it produces, gestational diabetes is caused when pregnancy hormones and hormones produced by the placenta lead to such an increase in the blood glucose level that the pregnant woman’s pancreas can no longer compensate for this by an increased insulin production. Adding this information together with the coherence increasing elements in general (im Allgemeinen) and a special type of diabetes  (ein Spezialfall der Zuckerkrankheit) eliminates this incomprehensibility.

2. Some of the subjects do not know the (exact) meanings of the following terms used in the text without explanations: chronisch (TAP SF 74-78; TAP EK 26-29), Insulin (TAP SF 261-289; TAP EK 224-232), Inselzellen (cf. TAP YG 159-161; TAP NL 129-134; TAP JS 84-106 and 177-185; TAP EK 66-67 and 265-268), T-Lymphozyten (cf. TAP JS 267-283 and 652-655), Gestation (cf. RIP SF 534-540; TAP YG 133-134; TAP EK 329-331), Glukose (cf. TAP SF 303-312; TAP YG 200-206; TAP NL 232-233 and 243-246; TAP EK 135-162), and Körperzellen (cf. TAP JS 312-320). To solve this problem, these terms must either be left away or explained. They should be retained and explained if they are used in physician-patient conversations (cf. e. g. TAP NL 98-101), but can be deleted if this is rather unlikely and they are not needed again in the text (cf. RIP JS 609-616).

3. For YG the last sentence in the text “Dabei ist jedoch das Risiko für die spätere Entwicklung eines Typ 2 oder Typ 1 Diabetes [sic] stark erhöht.” (In this case, there is a considerably higher risk of developing a type 2 or type 1 diabetes afterwards.) gives rise to the question whether this refers to the mother or the child: “beim kind oder bei der mutter?” (TAP YG 34). That it refers to the mother can easily be made more explicit here.

4. For JS the original text does not make clear whether type 1 diabetes is caused only by a combination of all three factors mentioned in the text (i. e., first, genetic predisposition, second, influences from outside such as certain virus infections, and third, a disorder of the immune system) or whether it may be caused by one or two of these factors alone:

<<den optimierten Text lesend> als ursachen des typ eins diabetes gelten heute das zusammenwirken von erblicher veranlagung und äußeren faktoren (.) .h sowie einer fehlsteuerung des immuns> <<verächtlich> ja> des klingt aber a so;=wie is des da? (.) einer fehlsteuerung. ist das jetzt das zusammenwirken all dieser drei sachen oder nur dieser zwei sachen? (.) als ursache gilt das zusammenwirken von erblicher veranlagung und äußeren faktoren. <<verächtlich> oke.> UND einer fehlsteuerung (3.0) aha (.) ja. (.) alle drei wahrscheinlich (oder) .hh der typ zwei (.) <<p> ja dann passt das.> (3.0) <<pp> ah=fehlsteuerung (-) des immunsystems.> <<p> ah.> .hh das klingt aber sehr unglücklich; (2.0) ((gibt einen schnalzenden Laut von sich)) diabetes (.) typ eins kann durch (.) erbliche veranlagung (—-) ja weil (—) ah so (.) das kommt dann halt doch a (.) erkrankung raus oder wie (.) durch a viruserkrankung; hh (.) .hhh ((seufzt)) typ eins diabetes (.) kann durch (—-) erbliche veranlagung (.) <<p> sowie einer fehlsteuerung des immunsystems verursacht werden.> .hh hhh (4.0) hm:. (3.0) (—) <<p> diabetes> <<flüsternd> typ eins> <<p> kann> (4.0) durch das zusammenwirken von erblicher veranlagung (2.0) mit äußeren faktoren, (1.0) oder (—) ja das is alles (.) oder einer fehlsteuerung des immunsystems (.) i glaub so geht das. (TAP JS 493-516)

Although the combination of all three factors seems more plausible to her, she combines the factors by or in her optimized version which shows that she is still not sure. Since the factors which cause the disease are an important information for the patient, the text must be made more explicit here.

3.2 What makes the text hard to understand

1. Since Diabetes mellitus is a specialized term for which there is the more general and thus comprehensible expression Zuckerkrankheit in German, YG, SF, and EK feel that this more comprehensible designation should be added already in the title of the article. Thinking aloud SF says:

ja, da würde ich aber auf jeden fall einmal schon in der überschrift die deutsche übersetzung hineinbringen weil sonst kennt ma sich ja nicht aus was des is. (TAP SF 41-44, cf. also TAP YG 40-44).

This makes sure that the reader knows what the text is about from the beginning.

2. SF feels that the semantic relation between the general designation Zuckerkrankheit and its scientific synonym Diabetes mellitus might not become clear if the latter is simply put in parentheses without additional explanations, and therefore should be made explicit:

<<sich selbst beim Tippen diktierend> krankheit, in der folge diabetes mellitus>, eigentlich das brauch ma (—) gor nix so wer ma des machen, (—-) in der folge, nein das is keine gute idee (—) zuckerkrank (.) heit, des kommt a mal weg, (-) diabetes mellitus, (.) jawoll, na mach ma schon so, in der folge diabetes mellitus. (4.0) <<zustimmend> mhm> (—-) nein, das müsst man auch besser erklären, weil ich weiß das ja was das is, aber jemand der nicht latein und nicht griechisch kann weiß das nicht, (.) in der folge (—) der ha lateienischen oder griechischen Bezeichnung ( ) wahrscheinlich, nein wissenschaftlichen <<sich selbst beim Tippen diktierend> wissenschaftlichen bezeichnung so, diabetes mellitus,> (TAP SF 55-67)

She follows this maxim also when introducing other alternative designations such as juvenil, T-Lymphozyten, Alterszucker, and Gestationsdiabetes (cf. TAP SF 167-169, 219-223, 381-384, 432-437). EK does not comment on this, but she too makes the relation between Diabetes mellitus and Zuckerkrankheit explicit (“Diabetes mellitus ist der Fachausdruck für Zuckerkrankheit”).

3. YG wonders about the term Definition in the title because to her mind the text provides more information on diabetes mellitus than just a definition (cf. TAP YG 344-349), so that the term does not fit. Furthermore, one may object that the term Definition is a hard word for many readers, which is a second reason for taking it out. (In spite of her objections, YG takes the term over in her final version.) EK deletes the term Definition in her title, but does not comment on this.

4. The subjects YG and NL wonder whether “ist gekennzeichnet durch” is the correct expression in the definition of diabetes mellitus or could be replaced by a simpler verb. Both of them feel that the latter is the case. Thinking aloud NL says:

<<den Ausgangstext laut lesend> diabetes mellitus ist gekennzeichnet durch eine chronische erhöhung (.) des blutzuckers> (3.5) eigentlich ist dieses gekennzeichnet durch für mich (—) überflüssig (2.5) is es nur ausschließlich gekennzeichnet dadurch oder (-) is es tatsächlich eine chronische erhöhung des blutzuckers? (7.0) ich glaub das kann man schon vereinfachen <<sich selbst beim Tippen diktierend> diabetes mellitus (1.5) klammer noch mal zuckerkrankeit,> (4.0) tja, tippen sollt ma halt können .hh hh .hhh <sich selbst beim Tippen diktierend> ist eine (.) chronische erhö::hung des blutzuckers> (8.0) oder noch mal ganz anders rum, .hh bei diabetes mellitus ist der blutzucker chronisch erhöht? (TAP NL 44-56, cf. also 57-82; and TAP YG 568 ff.)

5. For YG the first sentence does not make clear whether the participle construction “verbunden mit dem Risiko für schwere Begleit- und Folgeerkrankungen” (combined with the risk for serious other diseases which accompany or follow it) describes necessary features of the term diabetes mellitus and thus forms part of its definition, or gives just additional information. Thinking aloud she comments on this:

<<sich selbst beim Tippen diktierend> erhöhung des blutzuckers, (.) verbunden mit dem risiko> (2.0) MO:MENT (3.0) oke=verbunden mit dem risiko des interess:iert mich hier eigentli net wirklich (5.0) verbunden mit dem risiko (3.0) aber die zuckerkrankheit is:t eigentlich nur eine chronische erhöhung des blutzuckers; (—-) UND wenn ma erhöhtn blutzucker::, (.) wenn sich der erhöht (.) dann:: .hh können schwere beglEIT und FOLgeerkrankungen (—) FOLgen (TAP YG 45-52).

In the retrospective interview, she comes back to this comprehension problem saying:

beim erstn satz hab i mi a bissl gwundert, (—) weil <<den Ausgangstext lesend> zuckerkrankheit (.) ist gekennzeichnet durch chronische erhöhung des blutzuckers verbunden mit dem RIsiko.> (—) is eigentlich (—) a:: (-) <<all> ahso> .h sog ma so- (.) i persönlich versteh des anders. für mi is zucker; zuckerkrankheit chronische erhöhung des blutzuckers (-) und danach (—-) also (.) beziehungsweise (.) durch die erhöhung (—) kommen begleit- und folgeerkrankungen. aber es is net (3.0) zuckerkrankheit is net (2.5) gleich (-) begleit- (.) und folgeerkrankungen. (RIP YG 568-578)

The information in the participle construction does not form part of the definition, which should be made clear in the optimized version.

6. Four of the five subjects (SF, YG, NL, EK) are amazed that the term Zuckerspiegel (sugar level) is used in the plural; they have only heard of der Zuckerspiegel in the singular and wonder whether there are several sugar levels:

DIE zuckerspiegel ((tippt auf der Tastatur ohne zu schreiben)) (3.5) mi: irritiert des. (2.0) wieso hei die do mehrere nei=guet (.) <<tippend> wenn die do mehrer hei, werdn ma do wohl mehrere ha:;> (3.5) oke: (TAP YG 220-223; cf. also TAP SF 320-329)

In fact, there is only one blood sugar level, so that the plural is wrong and must be changed into a singular. Even if there were several blood sugar levels, using the singular would be the option to be preferred in this context because a distinction between different sugar levels is not necessary in the text and wondering about the plural requires memory capacity which will then not be available for processing the central information on diabetes. If a differentiation were relevant, the plural should be introduced explicitly so that the reader need not wonder about it. YG opts for the plural because, as she says in the retrospective interview (RIP YG 498-507), the author of the text is an expert and should know what he is talking about. This is also the reason why SF uses the plural (cf. TAP SF 326-329). NL and EK prefer to use the more common singular (cf. TAP NL 264-268; TAP EK 209-210).

7. In the original version, information which belongs closely together such as a) the different designations of each type of diabetes, b) the age when type 2 diabetes occurs, c) the beginning and the end of gestational diabetes (cf. items No. 9 and 24 in [url=http://www.intralinea.it/parallel/eng_more.php?id=815_0_31_0_M%]Table 4[/url]), and d) the first mentioning of the destruction of the insulin producing cells and the detailed explanation of how they are destroyed and why are given at different places in the text. YG brings some of them together (a and c), however, without commenting on it in her TAP (‘information clustering maxim’). EK brings the information under a) together (TAP EK 69-71). NL explicitly states that she wants to bring the information when type 2 diabetes occurs together:

<<sich selbst beim Tippen diktierend> meist erst nach dem (6:0) meist erst nach dem vierzigsten lebensjahr,> (3.5) der sich meist erstmals (.) nicht erst nach dem vierzigsten lebensjahr bemerkbar macht (3.0) ((tippt)) (12.0) in den letzen (2.0) na ja wenn ma hier schon von den lebensjahren spricht (  ) bemerkbar nicht bemerkbar .hh bemerkbar macht, dann zieh ich da (—-) den letzten satz vor <<den Ausgangstext lesend> in den letzten jahren hat sich das alter beim ersten .h jedoch zunehmend nach unten verlagert,> (TAP NL 345-354)

SJ comments on d) saying:

<<sich selbst beim Tippen diktierend> beruht .hh auf einem (.) mangel an insu .hhh hh (—) infolge (-) einer zerstörung> .hhh hh (2.0) der sogenannten bet (2.0) ah das klingt glaub i komisch, wenn das da hinten so (.) .h der insulin produzierenden zellen (2.0) .hh ja aber wodurch werden die zerstört? (2.0) <<den Ausgangstext lesend> diese zellen gehören zur bauchspeicheldrüse und sind ein bestimmter typ der sogenannten inselzellen. am höchsten ist die neuerkrankungsrate bei kindern,> (2.0) ah=so (.) do unten steht des erst (.) ha? (TAP JS 58-68)

This comment, too, shows that information belonging closely together should be provided together.

8. Without mentioning this explicitly, YG seems to follow a parallelism maxim. This can be seen in her optimized version, where she tries to structure the sections on the three types of diabetes parallelly (age groups and designations, disease itself, causes, effects).[6] NL follows the parallelism maxim explicitly stating twice that she wants to structure the information on type 2 diabetes in the same way as that on type 1 diabetes:

typ zwei diabetes beruht auf einem verminderten ansprechen .hhh beim typ zwei diabetes sprich sprechen die körperzellen (2.0) weniger gut auf insulin an? (—-) is des gar zu umgangssprachlich? (1.5) außerdem hätt ich gerne die gleich struktur wie oben typ eins diabetes wird (1.5) verursacht durch und so weiter (TAP NL 318-323)

<<sich selbst beim Tippen diktierend> vermindert auf: insulin ansprechen.> (—-) so des wird jetz des gleiche wie vorher mit dem jugendlichen diabetes (5.5) typ zwei: diabetes ((tippt)) (5.0) macht sich (3.0) meist erst (5.0) typ zwei diabetes wird auch altersdiabetes oder (.) al wird auch als altersdiabetes oder alterszucker (.) bezeichnet, (—-) da er sich meist nach dem erst=nach dem vierzigsten lebensjahr erstmals bemerkbar macht (5.0) ja dann hab ich die parallele struktur wie oben, das gefällt mir besser so umgedreht oke. (TAP NL 332-341, cf. also 381-382)

Parallelism has been propagated by stylistics for a long time. For the reader, it increases the predictability of what comes next in a text and therefore is important for text comprehensibility.

9. In the original text, Beta-Zellen (beta cells) is introduced as the specialized term for Insulin produzierende Zellen (insulin producing cells). YG decides not to eliminate this alternative designation from the text because it might be used in physician-patient conversations (cf. TAP YG 152-156), but she introduces it only once and then - in contrast to the author of the original text - goes on using the more telling expression Insulin produzierende Zellen, which I consider a good decision.

10. Several of the subjects (SF, YG, NL) wonder about the noun Untergang (decline). YG laughs when she reads it (TAP YG e. g. 186-187); SF says: “sagt man wirklich den untergang?” (TAP SF 244-245), “untergang gefällt mir überhaupt nicht” (TAP SF 254). NL comments:

in folge davon kommt es zu einem unter <<f> untergang der insulin produzierenden zellen> (3.5) also des kann i ma a net gut vorstellen (.) untergang der (—-) zellen (3.0) da hab ich gleich diese assoziation dass (.) diese zellen ausschauen wie schiffe und irgendwie attakiert werden und dann .hh sinken; und untergehen. was kann damit gemeint sein?=is da irgendwas im wörterbuch? ((schlägt im Wörterbuch nach)) natürlich nicht. (TAP NL 174-182)

This unintentional foregrounding of linguistic elements takes away memory capacity necessary for processing the content on diabetes and therefore should be avoided and, as NL’s TAP shows, be replaced by a semantically more precise formulation. NL says:

also untergang des klingt für mich komisch. das führt zu einem verlust an, .h dadurch werden die insulin produzierenden zellen (2.0) geschädigt, oder beschädigt, oder werden sie wirklich zerstört? .hh das würd ich jetz gern noch an fachmann fragen (-) was dann wirklich mit diesen zellen passiert. (3.5) ob einfach die anzahl reduziert wird, oder ob sie beschädigt und daher funktionsunfähig sin, (—-) oder ob sie wirklich völlig zerstört werden. das müsste man noch mal fragen. (TAP NL 195-203)

11. The original text says that type 2 diabetes, also called Altersdiabetes (old-age diabetes), occurs after age 40 in most cases, which is the reason for its designation. In recent years, however, people have been affected by this type of diabetes at an ever earlier age. For YG age 40 is not really old: “<<len> vierzge isch jo (.) no net (.) speziell: (-) alt.>” (TAP YG 404-405). Therefore she feels that the designation is not motivated semantically and wonders whether type 2 diabetes occurred later than age 40 in the past and the age has gone down to 40 only in recent times:

ja wieso (schiebe dire ete dere mol) noch vierzge o:f wenn i doch noch (—-) i schätz vierzge, (2.5) isch des jetzt neu, oder isch des jetzt alt; (2.5) <<den Ausgangstext lesend> in den letzten jahren hat sich das alter beim (—) erschten auftretens nach (1.5) UNten verlagert> (—-) .hh (3.0) jo:; hhh (2.0) meist erstmals nach dem vierzigsten (3.5) isch eigentlich ((lacht)) idiotisch, wenn ma da scho a:lterszuckers=heit, mit vierzge; (3.5) äh:: blödsinn; (TAP YG 252-260)

Since YG is the only of the five subjects who had this problem, this is not taken into account in the optimized version.

12. The expression “nach Beendigung der Schwangerschaft” causes YG to laugh when she first reads it (cf. TAP YG 32), later on she reads it with a disgusted undertone (TAP YG 429-446). She does not comment, however, on what is wrong with it. NL comments on it explicitly:

<<den Ausgangstext lesend> in der regel verschwindet (—-) diese Form des diabetes (-) nach beendigung der schwangerschaft> (—) beendigung des klingt mir aber viel zu aktiv (—) natürlich die geburt is das logische ende der schwangerschaft aber (.) einfach so beendigen? beenden kamma schwangerschaft ja, das ende der schwangerschaft. (4.0) <<sich selbst beim Tippen diktierend> in der regel verschwindet diese form des diabetes (2.5)> nach beendigung nein, (.) nach ende ((tippt)) (4.0) oder einfach nach der schwangerschaft (1.5) oder nach der geburt (10.0) nja ende der schwangerschaft kann auch a tragischeres (-) resultat sein als geburt (—) wemma des kind verliert (5.0) also kamma nicht einfach (.) also wär das vielleicht nicht ganz eindeutig (.) das was damit gemeint is wenn ich geburt schreibe (6.5) <<den optimierten Text lesend> in der regel verschwindet diese form des diabetes> (-) nach ende der schwangerschaft (-) oder nach der schwangerschaft (5.0) oder eben nach der geburt aber des is (-) nicht ganz optimal. (3.0) ((schnalzt mit der Zunge)) mhm=hm=hm <<den optimierten Text lesend> tritt diabetes erstmals während der schwangerschaft auf, .hh spricht man von schwangerschafts- oder gestationsdiabetes. (3.0) diese form des (—) außerdem thema rhema (—) diese form des diabetes (.) an den satzanfang ((klickt)) ach blödsinn (2.0) diese form des (2.0) diabetes .h verschwi:ndet ((klickt)) in der (2.0) in der regel (.) nach ende (1.5) nja schreib ma nach ende der schwangerschaft ((tippt)) (.) <<sich selbst beim Tippen diktierend> (  )ngerschaft> schwangerschaft, (1.5) hh oder mit dem ende der schwangerschaft (—) nein, verschwindet in der regel nach ende der schwangerschaft, (TAP NL 414-443)

Her criticism is justified because the German verb beenden possesses the semantic features <controlled by one’s will>, <intentionally>, which may lead to the wrong interpretation that the author does not (also) refer to the end of a pregnancy marked by the birth of the child, but only to an end caused by abortion. Both ends are meant. To make this clear a more general formulation is needed such as nach der Schwangerschaft (after the pregnancy).

13. The author of the original text has a strong tendency to use nominal style, which the subjects seem to find hard to understand. They transform several nominal formulations into verbal ones: “chronische Erhöhung des Blutzuckers” (NL), “Neuerkrankungsrate” (YG), “Untergang der insulinproduzierenden Zellen” (SF, YG, NL, EK), “das Ansprechen der Körperzellen auf Insulin” (SF, NL, EK), “das Alter beim Auftreten des Diabetes” (SF, YG), and “das Risiko für die spätere Entwicklung eines Typ 2 oder Typ 1 Diabetes [sic]” (YG, NL, EK). SF transforms “infolge einer Zerstörung der insulinproduzierenden Zellen” into verbal style saying:

<<den Ausgangstext lesend> infolge einer zerstörung der insulin produzierenden zellen> ich würde des auflösen mit einem nebensatz. (TAP EF 107-9)

SF and NL also feel that “Neuerkrankungsrate” is a hard word for somebody who does not know it and also try to transform it into a verbal expression, but do not succeed (cf. e. g. NL TAP 140-149). SF therefore decides to split it up into “Rate von Neuerkrankungen” (cf. TAP SF 143-153).

SF and NL explicitly mention their avoid-nominal-style strategy:

ein vermindertes ansprechen auf körperzellen das müsst ich sicherlich auch anders auflösen weil das is vielleicht doch ein bisschen zu schwierig zu verstehn (.) .hhh (4.0) beim typ zwei diabetes (3.0) .h würd ich nicht (.) nominalisieren sondern vielleicht (.) eher verbalisieren (TAP SF 345-350)´

na ma könnt nominalstil noch auflösen (TAP NL 452-453)

14. SF and NL feel that the original text does not make clear that the sentence following the expression “Fehlsteuerung des Immunsystems” explains what is meant by it. Thinking aloud SF says:

<<sich selbst beim Tippen diktierend> fehlsteuerung des immunsystems> (21.0) .hhh <<ff>> das passt mir nicht. <<den optimierten Text lesend> <<pp>> als ursache des typ eins diabetes sieht heute die wissenschaft das zusammenwirken (3.0) des immunsystems> (.) doch (.) punkt. (9.0) hm immunsystems (.) <<den Ausgangstext lesend> bestimmte weiße blutkörperchen richten sich speziell gegen die betazellen> (5.0) .hh das würd ich auf jeden fall noch amal wiederholen (—) immunsystems, ich werde da hier auch einen doppelpunkt machen damit man weiß das hängt damit zusammen (TAP SF 209-219)

Text coherence should be improved here. Both SF and NL use the strategy of inserting coherence increasing elements (e. g., a colon after “Fehlsteuerung” in the case of SF and dabei in the case of NL) also in other places (cf. TAP SF 296-298, 323-324; TAP NL 182-189).

NL also feels that the logical relation between the two sentences “Die Zuckerspiegel im Blut steigen an” (The sugar levels in the blood increase) and “der Körper muss als Energiequelle sein Fettgewebe aufzehren” (the body must fall back on its fatty tissues as a source of power) does not become clear. Thinking aloud she says:

das versteh ich jetz nicht ganz <<den Ausgangstext lesend> ohne insulin kann jedoch glukose nicht mehr aus dem blut in die körperzellen (1.5) aufgenommen (—) und verwertet werden. die blutzuckerspiegel im (2.5) die zuckerspiegel im blut steigen an (.) und der körper muss als energiequelle> (5.0) ach so hh anstatt dass die energie aus dem blut gewonnen wird, (4.0) muss der körper sein fettgewebe aufzehren. (TAP NL 233-240)

<<sich selbst beim Tippen diktierend> blutzuckerspiegel (4.0) an, und der körper ((tippt)) muss> (2.0) stattdessen vielleicht noch dazu? (4.5) muss stattdessen (2.5) sein fettgewebe als energiequelle aufzehren. .h der körper muss <<sich selbst beim Tippen diktierend> stattdess:> (4.0) na ganz zufrieden bin i damit no net aber jetz schreib is mal. .h der körper muss stattdessen (—) <<sich selbst beim Tippen diktierend> sein fettgewebe als (1.5) energiequelle> (13.0) <<den optimierten Text lesend> dadurch steigt der blutzuckerspiegel an (—) und da (2.5) körper muss (—-) stattdessen> (-) irgendwie is der übergang da nicht ganz klar, wenn der zuckerspiegel (2.0) ansteigt,
(—) aber TROtzdem kann der körper (—-) das nicht nutzen und (3.0) muss deshalb sein fettgewebe aufzehren. (1.0) des is glaub ich gemeint <<den optimierten Text lesend> ohne insulin kann jedoch glukose nicht mehr aus dem blut in die körperzellen aufgenommen und verwertet werden.> (4.0) der blutzuckerspiegel steigt (—) also an (2.0) ohne verwertet werden zu könne und daher muss der körper sein fettge (.) webe aufzehren (3.0) so rum; aber wie formulier ich des am besten? (3.0) der blut:zuckerspiegel steigt (—-) an, (—-) doch trotzdem?, na des muss i amal sehen <<sich selbst beim Tippen diktierend> der blutzcker (.) spiegel steigt (-) an,> jetz hab ich schon wieder bultzuckerspiegel geschrieben <<sich selbst beim Tippen diktierend> blutzuckerspiegel steigt also an (3.0) an,> ohne als energie (3.5) <<sich selbst beim Tippen diktierend> kann jedoch nicht als energiequelle (—) quelle genutzt werden (5.5) und da körper (—) muss> (3.0) und der (2.0) daher muss der körper (3.0) <<sich selbst beim Tippen diktierend> und daher muss::: ((klickt mit der Maus)) der (2.0) körper> sein fettgewebe (4.0) ((tippt mit den Fingern auf der Tastatur ohne zu schreiben)) und der körper muss stattdessen (2.0) und der körper <<sich selbst beim Tippen diktierend> muss:: statt:dessen sein fettgewebe aufzehren> so gefällt mir das besser. <<sich selbst beim Tippen diktierend> gewebe aufzehren.> punkt. (—-) so (—) den vorigen satz (—) werma dann gleich mal (1.0) also <<den optimierten Text lesend> ohne insulin kann jedoch glukose der wichtigste einfache zucker nicht mehr aus dem blut in die körperzellen aufgenommen und verwertet werden .hh der blutzuckerspiegel steigt also an, kann jedoch nicht als energiequelle genutzt werden und der körper muss stattdessen sein fettgewebe aufzehren> .h also so versteh ich des das des damit gemeint ist aber (—-) des wär auch a frage die ma zur sicherheit noch mit an experten klären sollte; (TAP NL 269-314)

She solves the problem by making explicit that the body cannot use the glucose in the increased blood sugar level and therefore has to fall back on its fatty tissue instead, which, to my mind, is an excellent solution.

15. For EK the explanation of type 2 diabetes is not clear. She wonders “was bedeutet ein vermindertes ansprechen”? (TAP EK 307). In the retrospective interview she wants to ask an expert about this:

unter der frage was versteht man unter dem .h typ zwei diabetes genau und wodurch wird er hervorgerufen ist jetzt als vage; also das müsst ma noch einen experten fragen und (—) .hh klären; hh was genau der typ zwei diabetes ist. (RIP EK 470-474)

16. NL wonders why type 1 and type 2 diabetes are not mentioned in the usual order in the last sentence:

<<sich selbst beim Tippen diktierend> dIabetes (—-) typ> (—) warum is des (1.5) zuerst des typ zwei und dann des typ eins (2.0) warum steht da des (.) zuerst? (3.0) na ja wahrscheinlich weil (-) sobald eine frau im (1.5) gebärfähigen alter is, ist es wahrscheinlicher dass sie erst später an diesem altersdiabetes erkrankt (6.0) größer (4.0) ah, trotzdem find ichs irgendwie seltsam zuerst zwei und dann eins zu schreiben (4.0) ((tippt)) (4.0) ((tippt)) so. (2.0) ah (-) das risiko später an typ eins oder zwei diabetes zu erkrANken gehört da natürlich noch rein (TAP NL 456-465)

The unusual order leads to a foregrounding of this information as can been seen from NL’s reflections. If there is a reason for the unusual order, this reason should be given explicitly. If no reason is given, the usual order should be used.

4 Summary of the results

[url=http://www.intralinea.it/parallel/eng_more.php?id=809_0_31_0_M%]Table 3[/url][7] [8] gives an overview of the problematic elements in the source text, of how many subjects had problems with them, of the dimensions in the Karlsruhe comprehensibility concept to which each problem can be attributed, as well as of a few maxims and strategies used by the subjects to solve these problems.

From the criticism summarized in [url=http://www.intralinea.it/parallel/eng_more.php?id=809_0_31_0_M%]Table 3[/url] we can derive the criteria which the subjects must have applied during their text evaluation process (cf. Footnote 2 in Section 2.1). In her framework for the differentiation of criteria applied to comprehension evaluation, Baker (1985:156ff.) differentiates between the following seven criteria, which she derives from both theoretical and empirical work on the cognitive processes involved in comprehension and comprehension monitoring: “lexical criteria”, which refer to comprehensibility at the level of individual words; “syntactic criteria”, which refer to grammatical correctness within sentence boundaries; and “semantic criteria” (a more correct term would be textlinguistic criteria), which require “consideration of the meanings of individual sentences and the text as a whole” (Baker 1985:156). Since the last criterion is the most crucial to effective text comprehension, Baker subdivides it into five subcategories, which she describes as follows:

(1) propositional cohesiveness, checking that the ideas expressed in adjacent propositions can be successfully integrated; (2) structural cohesiveness, checking that the ideas expressed throughout the text are thematically compatible; (3) external consistency, checking that the ideas in the text are consistent with what one already knows; (4) internal consistency, checking that the ideas expressed in the text are consistent with one another; and (5) information clarity and completeness, checking that the text clearly states all of the information necessary to achieve a specific goal. (Baker 1985:156f.)

In the study described here, all seven criteria were applied, which proves the subjects’ high metacognitive competence during comprehension monitoring (cf. the research review of empirical studies on comprehension monitoring in Baker 1985). Lexical criteria were applied in the following cases in [url=http://www.intralinea.it/parallel/eng_more.php?id=808_0_31_0_M%]Table 2[/url]: 2, 5, 13, 14, 16, and 19. Syntactic criteria were applied in cases 6 and 17. Textlinguistic criteria were used in the following cases: 18 (“propositional cohesiveness”), 11 and 12 (“structural cohesiveness”), 8, 9, and 10 (“external consistency”), 1, 7, and 15 (“internal consistency”), as well as 3, 4, and 20 (“informational clarity and completeness”). To every item criticized by the subjects one of the criteria between which Baker differentiates was applied; no additional criteria were used. This may indicate that Baker’s framework comprises all relevant criteria. The fact that the whole range of criteria in Baker’s framework was applied as an indicator of both the subjects’ metacognitive and metacommunicative competence (cf. Baker 1989:28, 33f.) and the depth of text processing that reception for optimizing reverbalization requires. Experiments which require less text processing depth, such as the application of the plus/minus-method (cf. Pander Maat 1996:29), provide much less feedback. Pander Maat (1996:28) explains this as follows: “(A(s long as there is no need to use a certain piece of information, there is no need to comprehend at a more than superficial level.” The optimization task forces the subjects into deeper text processing; it forces them to “use” the information in the text.

The criticism summarized in [url=http://www.intralinea.it/parallel/eng_more.php?id=808_0_31_0_M%]Table 2[/url] also shows in detail where the text is not optimally skopos-adequate. It can be used to produce a version optimized on an empirical basis. Such a version is juxtaposed to the original version in [url=http://www.intralinea.it/parallel/eng_more.php?id=810_0_31_0_M%]Appendix A[/url]. In this optimized version, all elements criticized have been changed except for three items which were criticized by only one subject and in an unconvincing manner: T-Lymphozyten, Körperzellen, and the semantic motivation of Altersdiabetes. Ideally, this optimized version should again become the object of optimizing reverbalization, which is an iterative method, until no further deficiencies can be recognized.

When is a text optimally comprehensible then? - According to the Karlsruhe comprehensibility concept, ideal comprehensibility depends on six dimension of a text: simplicity, structure, correctness, motivation, concision, and perceptibility. The subjects’ criticism of the diabetes text and their maxims and strategies, of which only a few could be mentioned here, reveal what they consider to be the basic ‘ingredients’ of comprehensibility. A comparison of these ‘ingredients’ of comprehensibility with the six dimensions of the Karlsruhe comprehensibility concept reveals that there were no items of criticism that could not be attributed to one of the dimensions of the Karlsruhe concept. This shows that the comprehensibility concept underlying the Karlsruhe model seems to match the intuitive comprehensibility concepts of the subjects in the experiment. Furthermore, the comprehensibility dimensions of the Karlsruhe concept cover all the comprehension evaluation criteria between which Baker differentiates: Although there are no 1:1 correspondences between the six dimensions of the Karlsruhe concept and Baker’s seven evaluation criteria, all items criticized in the text can be attributed both to a dimension of the Karlsruhe concept and to one of Baker’s evaluation criteria.

5 Critical evaluation of the method employed and conclusions

The method employed has the disadvantage that the only text-external source of information that the subjects could use during the experiment was a TRANSLOG dictionary. This dictionary only contained those terms which according to my mind might have been problematic for the subjects. As the experiments showed, however, the subjects also consulted the dictionary for other terms which were not contained. Furthermore, the subjects did not only have terminological questions, i. e., questions which can be answered on the lexical level, but also questions that involved more complex knowledge clusters, which they would have liked to ask an expert. Since such an expert was not available during the experiment and since the subjects were not allowed to use other sources of information such as the Internet, we must assume that the optimization depth the subjects aimed at and were able to achieve is lower than it would have been with an expert and other resources of information available. The use of other sources of information may have led to further questions concerning the comprehensibility of the text. The options the subjects had for comprehension regulation (Baker 1985:155) after recognizing a comprehension problem had been limited.

To solve this problem, the research design can be optimized as follows: The subjects are allowed to use the Internet and all electronic as well as conventional resources which they consider to be useful for their task. Online searches are recorded using a screen-recording program (ClearView). To be able to determine what exactly the subjects read on the screen when searching the Internet, for example, a Tobii eye-tracker [9] is used.

On searches in conventional sources notes are taken by a supervisor. With this method, the problems described above can be solved, but it also leads to an enormous increase in the amount of data obtained (1.2 gigabytes of screen records for an experiment of 1.5 hours). The optimized research design is particularly useful in experiments which focus on comprehension regulation after comprehension problems have been recognized, i. e., in experiments concentrating on research processes and knowledge integration as they are necessary during text production and translation processes.

This time-consuming method certainly cannot be applied to documents produced in a commercial environment on a regular basis. However, results from empirical studies carried out by Schriver (1987) show that exposure of writers to the results of so-called protocol-aided revisions like the ones described in this article do not only help these writers to optimize the texts which were tested but also to improve their ability to anticipate their audience’s needs in general. As a consequence, optimizing reverbalization and analyzing the data which result from such experiments may be an effective method for training technical writers and other text producers in companies and institutions.

About the author(s)

Since March 2010, Susanne Göpferich is Professor of Applied Linguistics and Director of the Centre for Competence Development at the University of Giessen/Germany. From 2003 till then she was Professor of Translation Studies at the University of Graz/Austria, and from 1997 to 2003 Professor of Technical Communication and Documentation at the Karlsruhe University of Applied Sciences/Germany. She has published widely on text linguistics, specialized communication and translation, comprehensibility research, technical writing, as well as translation theory and didactics. Her current research focuses on translation processes.

Email: [please login or register to view author's email address]

•  Previous article
• Next article