Exploring Oral ProficiencyProfiles of Heritage Speakers ofRussian and SpanishElvira SwenderAmerican Council on the Teaching of Foreign Languages

Cynthia L. MartinUniversity of Maryland

Mildred Rivera‐MartinezIndependent Language Consultant

Olga E. KaganUniversity of California–Los Angeles

Abstract: This article explores the linguistic profiles of heritage speakers of Russianand Spanish. Data from the 2009–2013 ACTFL‐UCLA NHLRC Heritage LanguageProject included biographical information as well as speech samples that were elicitedusing the ACTFL Oral Proficiency Interview–computer and were rated according to theACTFL Proficiency Guidelines 2012–Speaking by certified testers. The goal of the studywas to better understand the multiple linguistic, educational, and experiential factors thatcontributed to the speaking proficiency of these heritage speakers as well as how thosefeatures affected the tasks and contexts in which the speakers could more appropriatelycommunicate in the language. The data illuminate the linguistic strengths andweaknesses of speakers within certain ranges and highlight those language features thatprevented the participants from being rated at the next higher level. The authors discussimplications for teaching and learning and make recommendations for both heritagespeakers and their instructors.

Key words: implementation and assessment, interpersonal and presentational speak-ing, oral proficiency, program design, program monitoring and assessment

Elvira Swender (DA, Syracuse University) is the Director of Professional Programsfor the American Council on the Teaching of Foreign Languages, White Plains, NY.Cynthia L. Martin (PhD, University of Pennsylvania) is Associate Professor ofRussian, University of Maryland, College Park, MD.Mildred Rivera‐Martinez (PhD, Stanford University) is an Independent LanguageConsultant in the Washington, DC area.Olga E. Kagan (PhD, Pushkin Russian Language Institute, Moscow) is Professor,UCLA Department of Slavic Languages and Literatures, and Director of the Title VINational Heritage Language Resource Center.Foreign Language Annals, Vol. 47, Iss. 3, pp. 423–446. © 2014 by American Council on the Teaching of ForeignLanguages.DOI: 10.1111/flan.12098

Foreign Language Annals � VOL. 47, NO. 3 423

IntroductionAmple anecdotal information exists aboutthe oral proficiency profiles of heritage lan-guage learners (HLLs), and it is sometimesassumed that HLLs are capable of accom-plishing a full range of linguistic tasksbecause of their authentic‐sounding lan-guage and apparent ease of communication.However, such assumptions are not sup-ported by the broad spectrum in actual pro-ficiency levels among HLLs—a range thatincludes those who understand but do notspeak the language, those who can effective-ly carry out an assortment of basic dailytasks, and those who can use the languageaccurately and appropriately across a rangeof sophisticated professional and personaltasks and contexts. Given the complexitiesassociated with teaching and evaluatingHLLs, this study was carried out by theACTFL in conjunction with the NationalHeritage Language Resource Center(NHLRC) at the University of California–Los Angeles (UCLA) to gather reliable dataabout the range and variety of profiles ofHLLs’ oral proficiency.

Review of LiteratureAssessing the proficiency of HLLs is a par-ticularly challenging task, and within theexisting literature, a number of assessmentmeasures have been considered. Fairclough,Belpoliti, and Bermejo (2010) reported thata large number of American universitiescontinue to use second language (L2) assess-ments as placement exams for incomingstudents in Spanish heritage language pro-grams. In addition, while self‐assessmentmay not provide the most robust measureby itself, many heritage language programsuse self‐assessment or self‐assessment incombination with oral interviews as a meansof placing students or as a method of pre-liminary screening for separating HLLs fromL2 students. Carreira and Kagan (2011),e.g., reported that of the 1,732 HLLs of 22languages in their study, almost 68% felt thatthey were advanced or close to native speak-ers in listening comprehension and 44% felt

they were advanced or native‐like in speak-ing; in comparison, a much smaller number(27 and 19%, respectively) answered thattheir reading and writing proficiencieswere advanced or native‐like (pp. 40–64).

Shortly after the development of theACTFL Oral Proficiency Interview (OPI)protocol and rating system, Valdés (1989)suggested that using the OPI to assess HLLsmay not be appropriate because the test wasoriginally designed to assess L2 learners whostarted learning the language from zero.Valdés also argued that OPI testers mayunfairly penalize speakers of nonstandardor stigmatized language varieties, as didDraper and Hicks (2000), who stated thatthe use of the OPI for Spanish HLLs “has thepotential to lead to the denigration of arating based on the variety of the languagespoken” (p. 32).

However, in 1995, Valdés also asserted(p. 322) that proficiency assessments forHLLs must provide information about therange of functions that can be successfullycarried out by bilingual speakers in differentcontexts and, in 2001, the Heritage LanguageResearch Priorities Conference Report con-cluded that existing “assessment instrumentsshould be analyzed and reviewed or new in-struments devised” (UCLA, 2001, p. 16) toassess the competencies of HLLs, includingoral competency. In an investigation of theoral proficiency of Russian HLLs for place-ment purposes, Kagan and Friedman (2003)concluded that the OPI could be effectivelyused in the case of Russian HLLs and byextension perhaps also in those languagesthat have fewer manifested dialects or dialec-tal differences. Similarly, Martin (2010) ar-gued that, since both the ACTFL proficiencyscale and the ILR (Interagency LanguageRoundtable) scale are based on functionalcommunicative ability and are totally inde-pendent of a specific curriculum or how orwhere a language was acquired or learned,standard instruments such as the OPI couldserve as an effective general assessment toolfor any adult learner, including HLLs.

However, although many foreign lan-guage programs use the OPI as a key element

424 FALL 2014

in the formative and summative assessmentof L2 learners, particularly candidates seek-ing teacher certification, a recent search ofthe literature found only five papers thatreported OPI data for HLLs. In the 2003study mentioned above, Kagan and Fried-man found that Russian HLLs who had notreceived formal schooling in Russian dem-onstrated oral proficiency between Interme-diate Low and Advanced (on the ACTFLscale). In this study, speakers’ breadth ofvocabulary, instances of code mixing, andspeech rate were also compared with thoseof native speakers and foreign languagelearners (FLLs). The rate of speech ofmany of the HLLs turned out to be equalto that of the native speakers, while only oneof the FLLs came close to a native‐likespeech rate. Conversely, only the least pro-ficient HLL had a rate in the same range asthe FLLs.

Ilieva (2012) reported data on practiceOPIs with Hindi speakers who were testedduring an OPI workshop. She compared thelanguage produced by HLLs and FLLswhose levels of proficiency were betweenIntermediate Mid and Intermediate High,according to ACTFL descriptions, and as-serted that the Hindi HLLs’ speech produc-tion was natural: “HLLs consistentlyproduced native‐like intra‐phrase articula-tion, accents and intonational contours”(p. 20) as well as faster speech rate thandid FLLs. As in Kagan and Friedman(2003) above, Ilieva argued that “the FLLstypically spoke at a lower speed than HLLs.”Ilieva’s most important contribution may bea detailed description of what made HLLs’speech samples difficult to rate: As the in-terviewer tried to establish the ceiling andpursue more difficult topics, the HLLs usedEnglish words with increasing frequency.However, since urban native Hindi speakersin India are bilingual and “exhibit a similartendency to code‐mix or resort to English,”it is difficult to know whether this was adisplay of native‐like competency or the re-sult of an inability to produce appropriatelanguage in higher‐register Hindi (Ilieva,2012, p. 26). In the third recent study,

Polinsky (2008b) also found that HLLs’speech rate was slower than that of nativespeakers but faster than FLLs’ speech rate ofsimilar proficiency.

Montrul (2013) noted that “the vastmajority of linguistic and psycholinguisticstudies have investigated the lower end ofthe proficiency spectrum, characterizing thenon‐target‐like linguistic abilities of HLLs asthe products of incomplete acquisition and/or attrition” (p. 17). The same can be saidabout both teaching and testing: The vastmajority of discussion of HL testing hasfocused on placement at the lower levelsof instruction. To date, the only article dedi-cated to advanced levels of proficiency (3/4 perthe ILR and Superior/Distinguished per theACTFL) of HLLs is by Davidson and Lekic(2013, pp. 88–114), in which the authorscompared the outcomes of HLLs and FLLswho began their overseas immersion pro-gram with an oral proficiency rating ofILR 2 (ACTFLAdvanced‐Low). After havingspent a year in Russia, 70 percent of HLLsincreased their proficiency to 4 (ACTFLDistinguished), and the remaining 30 per-cent increased their proficiency to 3 (ACTFLSuperior) on the ILR scale. Non‐HLs typi-cally attained between 3 and 3þ (ILR).

Thus, while much research on HL teach-ing has become available over the pastdecade (see reviews by Brinton, Kagan, &Bauckus, 2008; Malone, Peyton, & Kim,2014; Schwartz Caballero, 2014), the fivestudies cited above show that the literatureon the assessment of HLLs is not plentiful.Malone et al. (2014) noted the need to assessthe interpersonal, interpretive, and presenta-tional speaking skills of HLLs (p. 355) andidentified five challenges that “impede heri-tage language proficiency assessment”: (1)the absence of specialized proficiency stand-ards for HLLs due to the fact that “currentproficiency standards are designed for L2learners”; (2) the lack of consensus in defin-ing basic terms (for example, who is an HLLand how can language proficiency bedefined?); (3) the lack of understanding oflanguage variation; (4) the lack of assess-ment instruments for measuring various

Foreign Language Annals � VOL. 47, NO. 3 425

linguistic modes—such as presentational orinterpersonal domains—of HLLs; and (5)the scarcity of assessment‐based researchon HLLs, especially those who speak lesscommonly taught languages (p. 349).

Having established the validity and reli-ability of the OPI for assessing the skills ofHLLs, a number of researchers have used avariety of protocols across a variety of lan-guages to collect samples of heritage speech,then have used these samples to identifydeficiencies in morphosyntactic and dis-course features and thus determine the fea-tures that prevent a speaker from being ratedat the next higher level (Friedman &Kagan, 2008; He, 2004, 2006; Isurin, 2011;Isurin & Ivanova, 2008; Kagan and Dillon,2006; Karapetian, 2014; Montrul, 2008;Polinsky, 2008a, 2008b; Weger‐Guntharp,2006). Martin, with Swender and Rivera‐Martinez (2013), recently reported prelimi-nary findings from the ACTFL/NHLRC proj-ect to assess the oral proficiency of Russianand Spanish HLLs according to the 2012ACTFL Proficiency Guidelines, with thegoal of gaining a better understanding ofwhat prevents HLLs of these languagesfrom scoring higher on the scale. The fullreport from that study is the focus of thisarticle. It addresses the following researchquestions:

1. What are the strengths and weaknesses ofHLLs of Russian and Spanish at variouslevels of proficiency?

2. To what extent is there a correlation be-tween the proficiency levels and the lan-guage background and experiences ofHLLs of Russian and Spanish?

Methods

ParticipantsFor the purposes of this study, HLLs weredefined as those individuals who grew upspeaking a home language other than thedominant language of the country in whichthey lived (in this case, the United States)and who switched to that dominant lan-

guage (in this case, English) at an earlyage and received the majority of their edu-cation in an English‐speaking school whilecontinuing to use their HL in some contexts,most often in informal settings, e.g., at homeand in the community. All participants wererequired to be older than 18 and either cur-rently or previously enrolled at the postsec-ondary level as an undergraduate orgraduate student. Russian and Spanishwere the languages selected for this study,with the intention to add other languages inthe future. Participants also had to agree toprovide biographical information abouttheir education and their language back-ground, language preferences, and use ofthe HL; to have their oral proficiency as-sessed using the ACTFL OPIc;1 and to au-thorize the use of the resulting speechsample for research purposes. Those whocompleted all of the components of thestudy received a certificate documentingan official ACTFL OPIc rating of their oralproficiency in the HL and a check in theamount of $25.

Recruitment and ScreeningParticipants were recruited from differentregions of the United States via announce-ments on ACTFL’s Web media sources andads posted in the ACTFL trade magazine,The Language Educator, as well as at the 2009and 2010 ACTFL conventions. ACTFL test-ers, other ACTFL members, and representa-tives of secondary and postsecondaryinstitutions, language organizations, andother professional organizations with largeheritage populations were also invited torecruit students to participate in the study.

Potential participants were initially pre-screened for “heritage speaker” status using aqualification survey (AppendixA). The surveycontained questions about the participant’sage of arrival in the United States, currentage, the contexts in which the HL was used,and whether they studied or had studied thelanguage. The survey also asked for a self‐assessment of the participant’s language profi-ciency based on self‐assessment statements,outlined the scope of the study, inquired as

426 FALL 2014

to their willingness to participate in the study,and asked those interested to agree to theterms of the study. One hundred ninety‐fiveRussian HLLs and 593 Spanish HLLs re-sponded to the initial qualification survey.

This group of potential participants wasthen invited to complete a more detailedbackground survey (Appendix B), whichaddressed in greater detail the contexts inwhich participants learned and used the HL;whether they could read and/or write in theHL; if they had travel, study, or other expe-riences in a country or countries where thetarget language is spoken; and whether theyhad any formal education in the HL. Anadditional self‐assessment of linguistic abil-ity was also included in this survey.

Ninety‐six HLLs of Russian and 100HLLs of Spanish completed the backgroundsurvey; of these initial participants, 53 Rus-sianHLLs and 41 SpanishHLLs subsequentlycompleted all phases of the project, includingboth surveys, the candidate authorizationform, and the OPIc assessment.

Certified ACTFL raters were selected torate samples and complete evaluation forms.A team of researchers then conducted a de-tailed linguistic analysis of each speech sam-ple. The authors analyzed the findings of thelinguistic analysis in relation to the profi-ciency levels assigned to specific samples inthe aggregate, as well as against the back-drop of the demographic data collected fromeach participant, in order to identify patternsof strengths and weaknesses of speakerswith different proficiency profiles.

Russian Heritage DemographicsOut of a total of 53RussianHLLs, themajorityof the respondents (N¼ 26; 62%) identifiedthe Russian Federation as their country ofbirth, followed by the United States (N¼11; 26%) and Ukraine (N¼ 6; 14%). Otherbirth countries were: Austria, Azerbaijan,Belarus, China, Georgia, Kazakhstan,Moldova, Turkmenistan, the United King-dom, and Uzbekistan (1 participant each).

The majority of respondents (70%)were in the 18–25 age group at the time of

the study. Almost half (45%) immigrated tothe United States at ages younger than 10.Twenty percent were born in the UnitedStates. More than half (53%) of the respon-dents reported limited experience with for-mal studies in a Russian‐speaking country.The majority of the respondents (62%) re-ported that they were more proficient inEnglish, 17% considered themselves equallyproficient in both languages, and 11% con-sidered Russian their most proficientlanguage.

Respondents reported that the languagemost spoken on the job was English (53%),with 26% using amix of Russian and Englishat work. Some respondents (21%) reportedthat they did not work, and none identifiedRussian as the language most spoken on thejob. In general, the language spoken mostoften was reported as English (60%) or amix(34%). Only 6% reported Russian as thelanguage most spoken.

Spanish Heritage DemographicsOf the total 41 participants in the study, themajority (N¼ 28; 69%) were born in theUnited States, followed by Mexico (N¼ 6;15%). Other birth countries were: Argentina,Chile, Colombia, Dominican Republic, ElSalvador, Spain, and Venezuela (1 participanteach).

The majority of respondents (78%)were in the 18–25 age group at the time ofthe study. Of those not born in the UnitedStates, the majority (70%) immigrated atages younger than 10. Seventy‐two percentreported little or no experience with formalstudies in a Spanish‐speaking country. Themajority (59%) reported that they weremore proficient in English, 29% consideredthemselves equally proficient in both lan-guages, and 12% considered Spanish as theirmost proficient language. The Spanishgroup reported the language spoken athome as mostly mixed (46%), Spanishonly (39%), or English only (15%). MostSpanish respondents (46%) reported usinga mix of English and Spanish on the job,followed by 29% who reported English only,

Foreign Language Annals � VOL. 47, NO. 3 427

and 24% who did not work. No respondentsreported using only Spanish at work. Themajority (59%) of Spanish respondents re-ported speaking a mix of English and Span-ish most often, followed by 37% whoreported speaking only English. Only 4%reported Spanish as the language spokenmost often.

Procedures

SurveysBoth the qualification survey and the back-ground survey were hosted on a technology‐mediated survey platform. A link to the sur-vey sites was shared with participants, whocompleted the survey within a 30‐day re-sponse window.

The ACTFL OPIcOnce qualified as a HLL of Russian or Span-ish based on the survey results, participantswere assigned a login and password and thencompleted the OPIc in their HL. A fixedform of the OPIc was selected for this study,rather than a live OPI or an adaptive OPIc, toensure that all participants were asked torespond to identical prompts ranging fromthe Intermediate to the Superior level, pre-sented over the telephone in the same order,thereby to eliminate tester and content var-iables and to minimize any computer orbandwidth issues that individual partici-pants might have encountered. All sampleswere digitally recorded, archived on theACTFL/Language Testing International da-tabase, and made accessible to certifiedACTFL raters for assessment and for reviewby the researchers.

All samples were blindly double ratedby two ACTFL‐certified raters according tothe standard ACTFL OPIc rating protocol,taking into consideration the assessmentcriteria of function, content/context, texttype, and comprehensibility/accuracy acrossthe entire test sample. When the ratings oftwo raters agreed exactly (major level andsublevel), a final official OPIc rating wasassigned. In cases of discrepancies, samples

were arbitrated by a third rater and a finalOPIc rating was assigned.

OPIc Rater Review FormFollowing each OPIc rating, the rater(s)completed one of three versions of theOPIc Rater Review Form (Appendix C),based on the participant’s overall proficiencyrating. The three versions of this form in-cluded Intermediate, for samples rated In-termediate Low and Mid; Advanced, forsamples rated Intermediate High, AdvancedLow, and Advanced Mid; and Superior, forsamples rated Advanced High and Superior.On the review form, raters documented theirrationale for assigning a given rating andnoted the primary deficiencies that pre-vented the speaker from functioning at thenext higher level (e.g., ability to deal withtopics beyond self, precision of vocabulary,expectations for accuracy, lexical and struc-tural interference from English). The formfacilitated the comparison of individual sam-ples’ strengths and weaknesses and permit-ted the researchers to identify aggregatetrends among speakers who were rated ata given major proficiency level (Intermedi-ate, Advanced, and Superior).

Linguistic Sample Review FormA team of researchers who specialize in dis-course analysis analyzed the specific linguis-tic features of each speaker’s response to eachindividual prompt using the OPIc HeritageSpeaker Linguistic Sample Review Form(Appendix D). The analysis included a gen-eral evaluation of the speaker’s performanceon selected tasks associated with the level offull performance at a particular proficiencylevel, as well as an evaluation of the tasks inwhich the speaker showed evidence of break-down.2 The focus was specifically on thefeatures (e.g., fluency, accuracy, pragmaticcompetence, sociolinguistic competence,text organization) and linguistic deficienciesthat contributed to functional breakdownand therefore to the inability to sustain thecriteria of the next higher level. Researchers

428 FALL 2014

provided specific comments related to therating criteria and quoted examples fromthe OPIc to support their ratings.

AnalysisPossible relationships between OPIc ratingand demographic factors were exploredusing correlations between survey dataquestions and OPIc scores.

Results—Russian

Distribution Across the ProficiencyLevelsThe distribution of official OPIc ratings forthe 53 participants who completed the sur-vey and the OPIc is shown in Table 1.

Comparison of Self‐Assessments andOfficial OPIc RatingsRussian participants’ self‐assessments werecompared with their official OPIc ratings;data are presented in Table 2. The largestgroup self‐assessed in the Advanced range(42%), followed by those who self‐assessedin the Superior range (32%) and those whoself‐assessed as Distinguished (28%).

As shown, 77% of the participants whoself‐assessed at the Advanced level (17 of 22)correctly self‐assessed at the Advanced levelwhile only about one‐third of the partici-pants at the Intermediate or Superior levelsassessed their level of proficiency correctly.Fifteen participants self‐assessed as Distin-guished; however, although the OPIc doesnot assess the Distinguished level of perfor-mance, 13 of those 15 participants wereofficially rated Superior and two were ratedAdvanced High. Interestingly, all but one of

the participants who incorrectly assessedtheir level of proficiency overestimated theirproficiency.

Analysis of Samples RatedIntermediate Mid and IntermediateHighIn the Russian sample, five speakers receivedratings in the Intermediate range: one atIntermediate Mid (IM) and four at Interme-diate High (IH). While this sample was toosmall to be statistically relevant, and cautionshould be exercised when drawing any kindof inferences or conclusions, the followingobservations stood out as noteworthy for thespeakers in this range. For samples rated IMand IH, functional breakdown occurredwhen learners tried to perform the Ad-vanced‐level tasks and was evidenced byeither no attempt to perform the task orby initiating a response but not completingor unsuccessfully completing it. The resp-ondents were unable to maintain therequired paragraph‐length text type, whichis required at the Advanced level. Mostresponses were marked by English interfer-ence. Half were marked by lack of structuralcontrol and lack of appropriate vocabulary.

To better understand the linguisticbreakdown that prevented these IM andIH speakers from being rated at the Ad-vanced level, the researchers analyzed par-ticipants’ responses with respect to fourspecific Advanced‐level functions: past nar-ration, present narration, description on atopic beyond the personal or autobiographi-cal, and participants’ ability to successfullyhandle a situation with a complication. Ofthe four tasks, participants were the most

TABLE 1

Russian Ratings Across Proficiency Levels

Superior Advanced High Advanced Mid/Low Intermediate Mid/High

17 9 22 5

Foreign Language Annals � VOL. 47, NO. 3 429

successful when narrating in the past, per-haps because in Russian, the grammar asso-ciated with past narration (i.e., past‐tenseverbs) is relatively simple. Similarly, han-dling a situation with a complication didnot appear to be particularly challengingfor these speakers. The most challengingfunction for this group was the ability tonarrate and describe when reporting a cur-rent event: In particular, participants haddifficulty providing cohesive and organizedtext and meeting level‐appropriate expect-ations for grammatical accuracy, includingpragmatic and sociolinguistic competence.In addition, for participants who were ratedat the IM and IH sublevels, the internalorganization required for the developmentof the text type still presented challenges.While three of the five speakers in this ratingrange demonstrated near‐native pronuncia-tion, all five participants lacked the vocabu-lary that was needed to successfully speakabout content areas beyond the speaker’severyday or autobiographical experience.

Analysis of Samples Rated AdvancedLow and Advanced MidOf the total number of 53 Russian HLLs, 5were rated Advanced Low (AL) and 17 wererated Advanced Mid (AM). All of thesespeakers demonstrated sustained perfor-mance in all the prompts intended to elicitAdvanced‐level functions. Regarding thespecific types of breakdown that preventedthem from reaching the Superior level, most

participants (99%) were unable to producewell‐organized extended discourse, al-though most were comfortable speaking inparagraphs, the text type that is required atthe Advanced level. Raters indicated the lackof highly precise vocabulary (38%).

To better understand the sources of lin-guistic breakdown that prevented AL andAM speakers from being rated at the Supe-rior level, participants’ responses to threeSuperior‐level tasks—discussing a topicfrom an abstract perspective, supportingan opinion, and hypothesizing—were ana-lyzed in greater detail. When they attemptedto discuss a topic from an abstract perspec-tive, the majority (88%) initiated the taskbut were unable to complete the task at theSuperior level. Most (83%) avoided dealingwith the topic abstractly by dealing with thetopic from a concrete perspective and talk-ing about personal experiences.

When attempting the Superior‐leveltask of stating and supporting an opinion,the most frequent areas of weakness in theAL and AM groups were failure to addressthe task (94%) and failure to produce well‐organized discourse (88%). Again, most(73%) avoided dealing with the topic ab-stractly by reverting to concrete personalexperiences. Most (88%) demonstrated in-sufficiencies in their use of precisevocabulary.

When attempting the Superior‐leveltask of hypothesizing, all of the AL andAM participants failed to address the task,although 38% made an incomplete attempt.

TABLE 2

Comparison of Self‐Assessments and Russian Official OPIc Ratings

Total numberwho self‐assessedat eachproficiency level

Correctlyself‐assessed

Overestimatedskills

Underestimatedskills

Intermediate 3 1 0 2Advanced 22 17 4 1Superior 13 5 8 0Distinguished 15 0 15 0

430 FALL 2014

For 27%, the failure to use precise vocabu-lary contributed to breakdown and thus pre-vented a rating at the Superior level. Morethan half of those who avoided making ahypothesis (55%) were unable to producewell‐organized extended discourse.

According to the ACTFL guidelines,Advanced Low and Advanced Mid speakersare characterized by their lack of control ofthe Superior‐level functions. In the case ofthe Russian HLLs’ samples, functionalbreakdown occurred when participants at-tempted to perform tasks at the Superiorlevel either because the speakers revertedto concrete examples drawn from personalexperience rather than addressing the issueor because the speakers were unable to dealwith the topic from an abstract perspective,support an opinion, and/or hypothesize. Inall cases, speakers reverted to Advanced‐level performance, which was manifestedwhen they reduced their text type to theparagraph level and treated the topicconcretely and anecdotally.

Analysis of Samples Rated AdvancedHighOf the Russian group of HLLs, nine wererated Advanced High (AH). In this study,the AH speakers were analyzed separatelyfrom AL and AM speakers precisely becauseAH speakers are more closely associatedwith the Superior level and are defined notin terms of performance at the Advancedlevel but rather in terms of their breakdownfrom the Superior level. Thus, the research-ers looked for evidence from the same threetasks—discussing a topic from an abstractperspective, supporting an opinion, and hy-pothesizing—to determine if participantswho were rated AH evidenced areas of defi-ciency that were different from those of theAL and AM groups.

Data showed that, when attempting todiscuss a topic from an abstract perspective,all AH speakers failed to address the task atthe Superior level. Of those who partiallyaddressed the task, most reverted to sharingpersonal experiences rather than systemati-

cally and abstractly addressing the issue. Asmaller number failed to use highly precisevocabulary (42%). A majority (57%) lackedthe ability to organize their responses inextended discourse. When they attemptedto support an opinion at the Superior level,the majority (86%) were unable to completethe task at the Superior level. Most avoideddealing with the topic abstractly by dealingwith the topic from a concrete perspectiveand talking about personal experiences. Thetext type presented a significant challengewith this task: Most participants (71%) wereunable to produce well‐organized extendeddiscourse or use precise vocabulary. Whenattempting to hypothesize at the Superiorlevel, participants rated AH were still unableto handle the task functionally. Only 42%initiated a response that addressed the task,but they were not able to complete the task.More than 70% failed to use precise vocabu-lary adequate to the topic/task.

Results—Spanish

Distribution Across the ProficiencyLevelsThe distribution of official OPIc ratings forthe 41 participants who completed the sur-vey and the OPIc is shown in Table 3.

Comparison of Self‐Assessments andOfficial OPIc RatingsSpanish participants’ self‐assessmentswere compared with their official OPIcratings; data are presented in Table 4.The largest group of Spanish respondentsself‐assessed in the Superior range (53%),followed by those who self‐assessed in theAdvanced range (39%), with only two re-spondents who self‐assessed in the Inter-mediate range. None self‐assessed asDistinguished. One participant’s self‐as-sessment was not reported. The HLLswho self‐assessed as Advanced were themost accurate in their assessment, whilethose who self‐assessed as Superior werethe least accurate.

Foreign Language Annals � VOL. 47, NO. 3 431

As shown, both of the two participantswho self‐assessed as Intermediate were ratedin the Intermediate‐level range. Eighty‐eightpercent of the participants (14 of 16) whoself‐assessed at Advanced were rated in theAdvanced range, while two overassessedthemselves and were officially rated in theIntermediate range. Only 2 of the 22 partic-ipants who self‐assessed at Superior wererated in the Superior‐level range: Twenty(91%) overassessed themselves and wereofficially rated in the Advanced range.

Analysis of Samples Rated IM and IHAs shown in Table 3, in the Spanish sample,four speakers received ratings in the Inter-mediate range: three at IM and one at IH.These intermediate‐level speakers werecharacterized by English interference intheir vocabulary and pronunciation. Errorsaffected their message and made it difficultfor interlocutors to understand. Compre-hensibility required a sympathetic listeneraccustomed to learners or nonnatives for thecommunication to be successful.

The features of the heritage Intermediate‐level speakers were remarkably similar tothose of true L2 learners in terms of theiremerging development of the functionsassociated with the Advanced level; that is,their speech samples lacked structural con-trol, exhibited limited vocabulary, and werecharacterized by an inability to performthe task. On the other hand, the speech ofthe HLLs in this study was marked by morefluency and confidence, and more ability tofill cultural and linguistic gaps than what isgenerally associated with Intermediate‐levelL2 learners.

To better understand the linguisticbreakdown that prevented the IM and IHspeakers from being rated at the Advancedlevel, the researchers analyzed participants’responses with respect to four specificAdvanced‐level functions: past narration,present narration, description on a topicbeyond the personal or autobiographical,and participants’ ability to successfullyhandle a situation with a complication. Aswith the Russian samples, when dealing with

TABLE 4

Comparison of Self‐Assessments and Spanish Official OPIc Ratings

Total numberwho self‐assessedat eachproficiency level

Correctlyself‐assessed

Overestimatedskills

Underestimatedskills

Intermediate 2 2 0 0Advanced 16 14 2 0Superior 22 2 20 0Distinguished 0 0 0 0

TABLE 3

Spanish Ratings Across Proficiency Levels

Superior Advanced High Advanced Mid/Low Intermediate Mid/High

2 11 24 4

432 FALL 2014

Advanced‐level probes, the Spanish HLLsinitiated a response to the task at the Ad-vanced level, but the response was eitherincomplete or unsuccessful. Speakers wereunable to sustain performance at theAdvanced level.

Of the four tasks, participants who wererated IM or IH most often experienced lin-guistic breakdown when attempting tospeak about a current event. Providing de-tail, explanation, and description related tothe event beyond personal and autobio-graphical experience proved most challeng-ing. These Intermediate‐level speakers weremore at ease dealing with topics related toeveryday routines and familiar topics. Alllacked the vocabulary to engage in conver-sations on topics that went beyond theireveryday and autobiographical experiences.

The production of cohesive and orga-nized text was the least controlled feature inthe performance of these speakers. Lackingwere the connectors and internal organiza-tion needed to produce paragraph‐lengthdiscourse. The ability to handle a situationwith a complication proved to be less chal-lenging for this group. The most successfultask for the Intermediate HLLs was the pastnarration. They were more successful in us-ing past‐tense verb forms, aspect, and irreg-ular structures than what is typicallyexpected of L2 learners at this level.

Analysis of Samples Rated AL and AMOf the total 41 Spanish HLLs in the study,6 were rated AL and 18 were rated AM.These speakers demonstrated sustained per-formance in all Advanced‐level functionswith the appropriate text type and accuracyfor the level.

The most evident weaknesses of thisgroup that prevented a rating at the Superiorlevel were the inability to organize and pro-duce extended discourse, the lack of com-municative strategies to deal with the topicabstractly, and the lack of precise vocabu-lary. When they attempted to discuss a topicfrom an abstract perspective at the Superiorlevel, producing extended discourse pre-

sented a serious challenge: Participantswere unable to connect ideas and producelanguage that was internally well organized.The majority (86%) were not able to dealwith the topic from an abstract perspective.Most initiated the task at the Superior levelbut were unable to complete it. Limitationsin the use of precise vocabulary (77%) alsohindered their ability to handle the topicfrom an abstract perspective. Similarly,when they attempted to support an opinion,their most frequent area of weakness wasonce again in producing well‐organized dis-course (91%). Half of the group (50%) failedto address the task from an abstract perspec-tive and instead reverted to concrete exam-ples, with a smaller group resorting topersonal experiences (27%). A commonchallenge in dealing with this task was thelack of precise vocabulary (64%). Finally,when attempting to hypothesize at the Supe-rior level, none of the AL and AM speakersaddressed the task at the Superior level, andnone dealt with the topic abstractly.

As in the Russian group, Spanish ALand AM HLLs were characterized by theirlack of control of the Superior‐level func-tions. When participants attempted to per-form tasks at the Superior level, functionalbreakdown occurred either because thespeakers reverted to examples from personalexperience rather than addressing the issueor because the speakers were unable to dealwith the topic from an abstract perspective,support an opinion, and/or hypothesize. Inall cases, speakers reverted to Advanced‐level performance and reduced their texttype to the paragraph level or treated topicsconcretely or anecdotally.

Analysis of Samples Rated AHSpanish HLLs who were rated AH showedthe ability to perform at the Superior levelmost of the time, but they were unable tosustain that performance across the range offunctions and topics and thus meet expect-ations at the Superior level. The researcherschecked whether speakers reverted to exam-ples of personal experience rather than

Foreign Language Annals � VOL. 47, NO. 3 433

addressing the issue, or if they were unableto deal with the topic abstractly. When de-scribing breakdown at the Superior level,raters indicated deficiencies related to failureto use highly precise vocabulary, errors instructural control of low‐frequency struc-tures and discourse organization, andincomprehensibility of the content of themessage.

The primary reason for AH instead ofSuperior ratings in the case of the SpanishHLL group was functional rather than struc-tural breakdown. Speakers showed limitedability to develop abstract ideas while elabo-rating on and creating internally cohesivemessages. Patterns of error were similar tothose made by L2 learners. For example,speakers lacked control over the linguisticformulations that allow them to speculateand elaborate on outcomes and consequen-ces, e.g., the use of the past subjunctive andother complex grammatical structures.

As in the Russian sample, researcherslooked for evidence from three tasks—dis-cussing a topic from an abstract perspective,supporting an opinion, and hypothesizing—in order to determine the areas of deficiencyof the participants who were rated AH.When attempting to discuss a topic froman abstract perspective, the majority failedto address the task at the Superior level(86%). They partially addressed the taskbut reverted to examples from personal ex-perience rather than addressing the issueand showed insufficiencies in their use ofprecise vocabulary. A majority lacked theability to organize their responses in extend-ed discourse (72%) despite evidence of pre-cise vocabulary.

When they attempted to support anopinion at the Superior level, none of theSpanish participants in this study were ableto deal with the topic abstractly. Most (86%)were unable to address the task and to con-trol communicative strategies to organize aresponse in extended discourse. Finally,when they attempted to hypothesize at theSuperior level, 38% failed to address thetask; 71% were unable to deal with the topicabstractly, and all lacked the ability to orga-

nize their response in extended discourse. Anumber of participants (43%) lacked theprecise vocabulary required to address thetasks appropriately.

Comparing Heritage Profiles WithOfficial RatingsIn addition to observing specific linguisticfeatures of the heritage participants by level,the researcher conducted analyses to deter-mine if any of a number of heritage profilefeatures, (i.e., age of immigration, formalstudies in the HL, use of the HL at home,and the ability to read and write in the HL)were associated with higher (Advanced andSuperior) levels of proficiency.

In considering the strengths and weak-nesses of Russian and Spanish HLLs at vari-ous levels of proficiency, the data suggesteda relationship between explicit/formal lan-guage instruction at the college level andhigher levels of proficiency. In the Russiangroup, there appeared to be a relationshipbetween higher ratings and formal schoolingin a Russian‐speaking country. In the Rus-sian HLL group that attained a rating of ALor higher, 28 speakers had formally studiedin a Russian‐speaking country and 24 hadstudied Russian formally in the UnitedStates. The majority (85%) of RussianHLLs identified the Russian Federation (orthe former Soviet Union) as their country ofbirth. Only 15% identified the United Statesas their country of birth. In the case ofSpanish, most speakers were born and raisedin the United States, 13 had formally studiedin a Spanish‐speaking country, and 36 hadstudied Spanish formally in the UnitedStates. In both cases, formal language in-struction (even in the United States, but atthe college level) was a commonality amongthose participants who achieved ratings inthe Advanced range and higher (seeFigure 1).

The number of candidates who reportedtheir ability and practice of reading (47) andwriting (46) in Russian was another com-monality among the Russian HLLs who at-tained a rating of AL or higher. Equally, the

434 FALL 2014

number of candidates who reported theirability and practice of reading (37) and writ-ing (37) in Spanish suggested a similarconclusion.

Additional demographic features ap-peared to have a positive relationship tohigher levels of spoken language proficien-cy in the HL. These included age of arrivalin the United States3 and language usedmost often (see Table 5). Interestingly,travel to a country where the target lan-guage is spoken appeared to have a mod-erate negative relationship to higher levelsof proficiency.

The data also suggested some trends interms of the HLLs’ language use outside theclassroom. Of all the participants in thestudy, only a few in the Spanish groupindicated that Spanish was the primary lan-guage used at work. In all other cases,speakers in both groups indicated thatthey used a mix or English only at homeand with friends.

DiscussionThe first research question addressed thestrengths and weaknesses of HLLs of Rus-sian and Spanish at various levels of profi-ciency. Research has shown that the profilesof HLLs differ from the typical profiles ofboth L2 learners and native speakers. How-ever, such qualities as native‐like fluencyand pronunciation do not compensate forlack of sustained functional ability as de-fined by the Guidelines. HLLs’ confidence,fluency, and comprehensibility when speak-ing among peers, at home, or in the heritagecommunity are not sufficient indicators of ahigh level (Superior) of professional profi-ciency. Furthermore, overconfidence on thepart of HLLs or the instructor as a result ofthese features may possibly hinder learners’motivation to work systematically to im-prove linguistic skills.

Data showed that HLLs of Russian wereonly modestly successful in estimating theirlevel of language proficiency (23 of the

FIGURE 1

Relationship of Higher Proficiency Levels and Formal Study of HL

Foreign Language Annals � VOL. 47, NO. 3 435

53 participants, 43%). Interestingly, partic-ipants who self‐assessed at the Advancedlevel were more accurate than those at theIntermediate level, and all the participantswho self‐assessed as Distinguished were in-correct. Similarly, in Spanish, 18 of 40 par-ticipants (45%) were accurate in their self‐assessment, and those who self‐assessed atAdvanced were the most accurate. This isinteresting in that it confirms the generaltendency of HLLs to overassess their abili-ties. This may be due in part to the ease withwhich these speakers can use their HL ininformal contexts language, i.e., with familyand friends, as well as their lack of fami-liarity with the requirements of functio-ning at higher levels of spoken languageproficiency.

Among the many findings, perhaps themost interesting come from the analysis ofthe transition from the Advanced to theSuperior level. Advanced‐level speakers ofRussian as a group demonstrated a varietyof levels of control when attempting Supe-rior‐level tasks. As expected, AH speakersshowed better discourse organization thanAL/AM speakers and also demonstratedless frequent or marked breakdown thanAL/AM speakers when dealing with a topicabstractly and when attempting hypotheses.Nevertheless, participants who were ratedAH inRussian evidenced the same functional

breakdown as participants who were ratedAL/AM: The general pattern across all mem-bers of the Advanced‐level group was torespond to the demands of Superior‐levelfunctions by either initiating a responseand then lowering the level of their discourseto the Advanced level by sharing personalexperience, anecdotes, and examples with-out addressing the abstract nature of thetopic, or by diverting their ideas to less com-plex concepts without returning to the ab-stract treatment of the question.

Similarly, while responding to Superior‐level tasks, Advanced‐level Spanish HLLsshowed a variety of levels of control overthe use of precise vocabulary and communi-cative strategies to express themselves. AHspeakers showed better communicative strat-egies in the organization of their discourseand in their attempt to address the Superior‐level tasks than those rated AL/AM. In addi-tion, AH speakers showed less evidence ofbreakdown across all Superior tasks (i.e.,dealing with topics abstractly, supportingan opinion, hypothesizing) than AL/AMspeakers. Similar to the Russian speakers,however, when prompted at the Superiorlevel, the Advanced‐level speakers of Spanishin this study often showed functional break-down: Superior‐level tasks (i.e., dealing withtopics abstractly and supporting opinions)were generally not addressed at the required

TABLE 5

Language Survey Topic Correlations With OPIc

Russian Spanish Combined

OPIc score 1 1 1Age of U.S. arrival 0.282c 0.439a 0.322b

Language used at home �0.019 0.547a 0.295c

Language used most often 0.04 0.212c 0.154Student status 0.255c 0.05 0.146Travel to country �0.078 �0.319e �0.224d

aStrong positive relationshipbModerate positive relationshipcWeak positive relationshipdWeak negative relationshipeModerate negative relationship

436 FALL 2014

level of abstraction; speakers in this studytended to divert the focus of their attentionto personal experience instead of addressingthe level of abstraction and complexity ex-pected at the Superior level. Their text typewas predominantly limited to the single‐par-agraph level. In addition, despite their generalcontrol of the grammatical features of thelanguage, the Spanish HLLs in this studyevidenced patterns of structural errors andincorrect linguistic formulations when at-tempting to speculate and elaborate on possi-ble outcomes and consequences, e.g., whenthe use of subjunctive mood and conditionaltenses and structures for which there is nodirect parallel structure in English wererequired.

The data suggest that a number of fac-tors are associated with HLLs’ proficiencyscores. First, there appears to be a relation-ship between explicit/formal language in-struction and higher levels of proficiency,confirming a misperception that HLLs “donot need” formal instruction because theyalready “speak the language.” In addition,the age of arrival in the United States and thelanguage used most often at home wereshown to be strongly predictive of higherproficiency ratings.

Taken together, the findings suggestthat the communicative strategies that arepermissible when interacting in informalcontexts, such as mixing languages, mayactually hinder speakers whowish to achievehigher proficiency levels. The habits that areacquired when using a mix of languagesinformally do not appear to keep someonefrom achieving a rating in the Intermediaterange, since comprehensibility at that levelmay still require a “sympathetic interlocu-tor.” English interference observed as a char-acteristic in Advanced‐level speakers did nothinder communication to the level of incom-prehensibility when HLLs were speaking ininformal contexts. However, this interfer-ence will cause miscommunication and/ordisturb the educated speaker in more formaland professional contexts and will prevent aspeaker from achieving a rating of Superior.If communicative patterns that are accept-

able in Intermediate‐level tasks become fos-silized patterns (learned or acquired), thesepatterns will affect the speaker’s ability tofunction effectively in professional contextsin a monolingual environment.

The second research question investi-gated the extent to which there were rela-tionships between the proficiency levels andthe language background and experiencesof Russian and Spanish HLLs. Data suggestthat the use of the HL predominantly ininformal contexts, i.e., with family andfriends, together with the lack of use ofthe language in professional and formalcontexts, may explain the difficulty thesespeakers encountered when attempting toexpand communication beyond casual con-versations and references to personal expe-riences. When prompted to speak abouttopics beyond self and their immediate en-vironment, e.g., health policy, environmen-tal issues, technology, or politics, theparticipants tended to revert to personalexperiences. When prompted to discusstopics from an abstract perspective or todiscuss issues, the participants tended tohandle the topics concretely or provide an-ecdotal information.

Implications for InstructionBased on thesefindings, a number of strategiesand instructional practices can be recom-mended to support the development of higherlevels of proficiency for Russian and SpanishHLLs. Clearly, educators cannot influence theage of immigration of HLLs nor the amount oftime spent using the target language outsidethe classroom in home, community, or pro-fessional settings. Educators can, however,impact the quality of formal instruction pro-vided for heritage learners.

Moving From Intermediate toAdvancedThe few candidates rated Intermediate in thisstudy showed emerging development of thefunctions associated with the Advancedlevel; that is, they demonstrated lack of struc-tural control, limited vocabulary, and the

Foreign Language Annals � VOL. 47, NO. 3 437

inability to perform the task. A betterunderstanding of what is required to reachthe Advanced level as described in theACTFL Proficiency Guidelines will serveto help instructors frame curriculum andlearning experiences for speakers at the In-termediate level so as to identify and targetappropriate goals for reaching the Ad-vanced level. HLLs at the Intermediate levelresembled true L2 learners in terms of theirlexical limitations, limited structural con-trol, and inability to consistently producecohesive, paragraph‐length discourse.While native‐like pronunciation and flu-ency, as well as a high degree of confidence,can help their progress toward the Ad-vanced level, Intermediate‐level HLLsneed to expand their vocabulary to go be-yond the self and to use connectors to de-velop the internal cohesiveness andorganization of the message in order toconsistently produce paragraph‐lengthdiscourse.

To improve proficiency, the datasuggest the need for formal languageinstruction. The following strategies and in-structional practices are likely to help HLLsprogress from Intermediate to Advanced:

• Diagnostic feedback sessions to better un-derstand the areas where improvement isneeded

• Targeted practice on narration and de-scription in all time frames, participationin conversations about topics beyond thepersonal and familiar experiences, andlearning to organize interconnected dis-course of paragraph length

• Activities that systematically allow for theexpansion of content domains (vocabu-lary) and that go beyond the self andpersonal experience

• Targeted grammar instruction to elimi-nate patterned errors that often fossilizeand hinder performance beyond the In-termediate level, focusing particularly onword order, agreement, and connectors,and how to produce internally cohesiveparagraph‐length discourse when narrat-ing and describing in detail

• Frequent and sustained interactions withindividuals who speak only the HL, andnot English as well, so as to develop en-hanced awareness of how code‐switching(English interference) negatively impactscomprehensibility when communicatingwith monolingual speakers of Russian orSpanish. While code‐switching may allowHLLs to communicate in a heritage speak-ing environment within a specific regionof the country, the interference of anotherlanguage will reduce the comprehensibil-ity of interlocutors who are not familiarwith English, or the vocabulary that hasbeen switched.

Moving From Advanced to SuperiorThe great majority of Advanced‐level HLLsin this study overassessed at the Superiorlevel. This points to the need for speakersat this level to have a better understanding ofthe functional requirements at the Superiorlevel, along with the other assessment crite-ria associated with the level. Familiarizationwith the description of the Superior level inthe ACTFL guidelines can be a useful tool toassist instructors and learners in under-standing the rather more complex expecta-tions for language use, including the basicunderstanding that the Superior level repre-sents a level of proficiency that is equal to thefunctional ability of an educated speaker ofthe language. In order to reach a Superiorlevel rating, speakers must demonstrate theability to deal with topics abstractly, supportopinions, and hypothesize using preciserather than generic vocabulary in a varietyof formal and informal contexts and withoutpatterns of errors.

To improve proficiency, the data in thisstudy suggest that instruction for Advanced‐level speakers who seek to function at theSuperior level should focus on the following:

� Diagnostic feedback to better understandthe areas where improvement is needed

� Systematic practice in discussing issuesand their implications from both concreteand abstract perspectives so as to elevatespeakers’ language to the required level of

438 FALL 2014

sophistication and abstraction when at-tempting Superior‐level functions

� Familiarization with discourse strategiesand communicative devices used to sup-port opinions and hypothesize onoutcomes. Even at Advanced High, speak-ers in this study tended to initially addressthe function by stating an opinion butthen tended to revert to a concrete per-sonal example and not return to the func-tion or issue at hand. Learning to developformal discourse will not only increasetheir proficiency, but will also motivateand challenge HLLs beyond their personallevel of confidence and toward an appre-ciation of their own control of thelanguage.

� Systematic practice in constructing cogentarguments in extended discourse to sup-port opinions and speculate on possibleoutcomes and consequences (hypo-thesis), including the use of strategiesfor elaboration and the use of transitiondevices to organize these arguments witha high degree of accuracy

� Practice speaking about a wide rangeof general topics with the objective ofdeveloping an expanded level of precise,sophisticated vocabulary as well as devel-oping discourse and sociocultural strate-gies to communicate in professional andformal contexts

� Targeted grammar instruction to elimi-nate all patterned errors. Such patternswill prevent speakers from sustaining per-formance at the Superior level. While im-peccable structure is not required to attainbaseline proficiency at the Superior level,patterned errors or errors that distractfrom the message should be eliminated

� Focus on minimizing English interferencethat can prevent speakers from function-ing at the Superior level, in terms of bothaccuracy and content domains. Data showthat English interference does not appearto be a significant impediment to reachingthe Advanced level, where strategies tocircumlocute adequately compensate forlexical weaknesses. Nevertheless, this in-terference becomes a pattern of weakness

when addressing the functions ofthe Superior level, as these mistakes dis-tract educated native speakers from themessage

� Frequent and sustained interaction withindividuals who speak only the HL andnot English as well. Immersion and studyabroad—not just personal travel—in acountry where the HL is the language ofhigher education should be made avail-able, based on findings fromDavidson andLekic’s study (2013) in which participa-tion in the Russian flagship capstone over-seas immersion program resulted in evenhigher increases in HLLs’ proficiencywhen compared with nonheritage learn-ers. In addition, designated coursesshould be designed for Advanced‐levelHLLs at the university level so as to focusthe instruction based on the requirementsfor Superior‐level proficiency as well astheir specific needs

Limitations of the StudyIn spite of the extensive publicity and re-cruitment efforts, the greatest challenge wasidentifying a large number of HLLs who metthe very specific criteria that were estab-lished for participation and who were will-ing to complete both surveys and have theiroral proficiency assessed. In addition, manymore HLLs who had higher levels of profi-ciency chose to participate in the study,resulting in a much smaller number of In-termediate‐level speakers. In the end, theauthors recognize that the sample size maybe insufficient to produce wide‐reachingconclusions about all HLLs. However, whilethe final number of participants in the studywas smaller than originally anticipated,commonalities, trends, and definite patternsof abilities did emerge, including patterns ofstrength and weakness at each of the majorlevels. Further efforts should be made to addto the sample so as to expand, confirm, oradjust the initial findings and to continue tobuild a database of HLLs’ samples across arange of languages on which to conductfurther research. To this end, a similar studyfor Chinese HLLs is in progress.

Foreign Language Annals � VOL. 47, NO. 3 439

ConclusionThis study has shown that ACTFL assess-ments of oral proficiency can be used toassess the functional ability of HLLs: Inpractical terms, a final rating is indicativeof the functions and other assessment crite-ria associated with the major level (i.e., con-texts and content, text type, accuracy) that agiven speaker can perform in a sustainedmanner. Furthermore, the sample of lan-guage that is obtained from an OPI or anOPIc, in addition to being rated according toa national and international standard, canalso serve as the basis for further, more fine‐grained analysis. This study identifiedstrengths and weaknesses of Russian andSpanish HLLs at various levels of proficiencyand suggests some relationships between theproficiency levels of those HLLs and theirlinguistic backgrounds and profiles. In ad-dition, the findings suggest a variety of in-structional strategies and practices that willsupport HLLs’ continuing language develop-ment. First, by examining the specificfeatures of a speaker’s strengths andweaknesses, language educators can use ef-fective diagnostic feedback to serve as thebasis for setting specific individual goals. Inaddition, by familiarizing HLLs and theirinstructors with the features, assessmentcriteria, and requirements for each majorlevel of spoken proficiency, instructionand continuing assessment can then targeteffective strategies for improving functionalability. Such diagnosis will allow instructorsand students to develop individualized“road maps” that will allow HLLs to system-atically address those areas that preventthem from sustaining the next higher levelof proficiency. Such road maps must includeboth in‐class experiences as well as interac-tions with a variety of interlocutors in moreformal contexts and across a range of con-tent areas that require more precise vocabu-lary, use of complex structures, andorganizational strategies for producing co-hesive and extended discourse. In additionto formal study of the language, profession-al‐level work experiences, including intern-ships in an environment in which the

language is spoken, can also contribute tothe development of proficiency at higherlevels. Instructional programs should be de-signed in a way that motivates HLLs to aimfor the Superior level and beyond by ad-dressing linguistic weaknesses and buildingon their strengths.

AcknowledgmentsThe authors wish to acknowledge the UCLANHLRC for the funding that supported thisstudy. Our sincere thanks to Troy Cox ofBrigham Young University; researchers IrinaDolgova (Yale University), Manel Lacorte(University of Maryland), Adrian Massei(Furman University), and Zhanna Vernola(University of Maryland); ACTFL raters Na-talya Coffman (independent language con-sultant), Suzanne McLaughlin (ChemeketaCommunity College, ret.), Ida Raynes (De-fense Language Institute‐Foreign LanguageCenter), Maria Shardakova (Indiana Univer-sity), and Joseph Weyers (College ofCharleston); and ACTFL research coordina-tors Natalie Boivin and Jeanmarie O’Leary.

Notes1. The ACTFL OPIc is a valid and reliable

assessment that emulates the “live”ACTFL OPI, but with prompts that aredelivered through a carefully designedcomputer program and via a virtual ava-tar. The goal of the OPIc is the same asthat of the OPI, i.e., to obtain a ratablesample of speech that can be evaluated bya certified rater and assigned an ACTFLrating according to the descriptions inthe ACTFL Proficiency Guidelines 2012–Speaking. The speech sample is digitallyrecorded, archived in a searchable data-base, and rated by certified ACTFL raters.According to a 2006 side‐by‐side valida-tion study, OPIc ratings are consistentwith OPI ratings (Surface, Poncheri, &Bhavsir, 2008).

2. Breakdown is defined as evidence thatthe speaker is no longer able to sustainthe criteria of the major level. It is dem-onstrated in a variety of ways: the

440 FALL 2014

inability to accomplish the task, avoid-ance of the task, or substitution of thetargeted task with a lower level. The de-crease in the type and quantity of dis-course produced, an increase in errors,the disintegration of language, memo-rized responses unrelated to the prompt,and/or silence are also indications ofbreakdown.

3. Age of arrival in the United States mayalso be related to number of years offormal instruction in the HL in the coun-try of the HL.

References

ACTFL. (2012). ACTFL Proficiency Guide-lines 2012—Speaking. Retrieved June 26,2014, from http://www.actfl.org/publications/guidelines‐and‐manuals/actfl‐proficiency‐guidelines‐2012.

Brinton, D., Kagan, O., & Bauckus, S. (Eds.).(2008). Heritage language education: A newfield emerging. Mahwah, NJ: LawrenceErlbaum.

Carreira, M., & Kagan, O. (2011). The resultsof the National Heritage Language Survey: Im-plications for teaching, curriculum design, andprofessional development. Foreign LanguageAnnals, 44, 40–64.

Davidson, D., & Lekic, M. (2013). The heri-tage and non‐heritage learner in the overseasimmersion context: Comparing learning out-comes and target‐language utilization in theRussian flagship. Heritage Language Journal,10, 88–114.

Draper, J. B., & Hicks, J. H. (2000). Wherewe’ve been; what we’ve learned. In J. B. Webb& B. L. Miller (Eds.), Teaching heritage lan-guage learners: Voices from the classroom (pp.15–35). Yonkers, NY: ACTFL.

Fairclough, M., Belpoliti, F., & Bermejo, E.(2010). Developing an electronic placementexamination for heritage learners of Spanish:Challenges and payoffs. Hispania, 93, 270–289.

Friedman, D., & Kagan, O. (2008). Academicwriting proficiency of Russian heritage speak-ers: A comparative study. In D. Brinton, O.Kagan, & S. Bauckus (Eds.), Heritage languageeducation: A new field emerging (pp. 181–198).Mahwah, NJ: Erlbaum and Associates.

He, A. W. (2004). Identity construction inChinese heritage language classes. Journal ofPragmatics, 14, 199–216.

He, A. W. (2006). Toward an identity theoryof the development of Chinese as aheritage language. Heritage Language Journal,4, 1–28.

Ilieva, G. N. (2012). Hindi heritage languagelearners’ performance during OPIs: Character-istics and pedagogical implications. HeritageLanguage Journal, 9, 18–36.

Isurin, L. (2011). Russian diaspora: Culture,identity, and language change. Berlin: DeGruyter.

Isurin, L., & Ivanova‐Sullivan, T. (2008). Lostin between: The case of Russian heritagespeakers. Heritage Language Journal, 6, 72–104.

Kagan, O., & Dillon, K. (2006). Russian heri-tage learners: So what happens now? Slavic andEast European Journal, 50, 83–96.

Kagan, O., & Friedman, D. (2003). Using theOPI to place heritage learners of Russian.Foreign Language Annals, 36, 536–545.

Karapetian, S. (2014). “How do I teachmy kidsmy broken Armenian?” A study of EasternArmenian heritage language speakers in LosAngeles (Unpublished doctoral dissertation).UCLA.

Malone, M. E., Peyton, J. K., & Kim, K. (2014).Assessment of heritage language learners: Is-sues and directions. In T. J. Wiley, J. K. Peyton,S. Christian, S. C. K. Moore, & N. Liu (Eds.),Handbook of heritage, community, and NativeAmerican languages in the United States:Research, policy, and educational practice(pp. 349–358). New York: Routledge/CAL.

Martin, C. (2010). Assessing the oral proficien-cy of adult learners, “heritage” and “native”speakers using the ILR descriptors and ACTFLguidelines: Considering the challenges. Rus-sian Language Journal, 60, 167–183.

Martin, C., with Swender, E., & Rivera‐Marti-nez, M. (2013). Assessing the oral proficiencyof heritage speakers according to the ACTFLProficiency Guidelines 2012–Speaking. Heri-tage Language Journal, 10, 73–87.

Montrul, S. (2008). Incomplete acquisition inbilingualism. Re‐examining the age factor. Am-sterdam: John Benjamins.

Montrul, S. (2013). How “native” are heritagespeakers? Heritage Language Journal, 10,15–39.

Foreign Language Annals � VOL. 47, NO. 3 441

Polinsky, M. (2008a). Heritage language nar-ratives. In D. Brinton, O. Kagan, & S. Bauckus(Eds.), Heritage language education: A new fieldemerging. Mahwah, NJ: Lawrence Erlbaum.

Polinsky, M. (2008b). Russian gender underincomplete acquisition. Heritage LanguageJournal, 6 [Electronic version]. RetrievedJune 29, 2014, from http://www.heritagelan-guages.org/

Schwartz Caballero, A. M. (2014). Preparingteachers to work with heritage language learn-ers. In T. J. Wiley, D. Kreeft Peyton, S. Chris-tian, et al. (Eds.), Handbook of heritage,community, and Native American languages inthe United States: Research, policy, and educa-tional practice (pp. 359–369). New York: Rout-ledge/CAL.

Surface, E. A., Poncheri, R., & Bhavsir, K.(2008). The ACTFL OPIc validation projecttechnical report. Retrieved June 30, 2014,from http://www.languagetesting.com/wp‐content/uploads/2013/08/ACTFL‐OPIc‐En-glish‐Validation‐2008.pdf

University of California, Los Angeles. (2001).Heritage language research priorities conferencereport. Los Angeles, CA: Author. RetrievedJune 29, 2014, from http://www.cal.org/heritage

Valdés, G. (1989). Teaching Spanish to His-panic bilinguals: A look at oral proficiencytesting and the proficiency movement. Hispa-nia, 72, 392–401.

Valdés, G. (1995). The teaching of minoritylanguages as “foreign” languages: Pedagogicaland theoretical challenges. Modern LanguageJournal, 79, 299–328.

Weger‐Guntharp, H. (2006). Voices from themargin: Developing a profile of Chinese heri-tage language learners in the FL classroom.Heritage Language Journal, 6, 29–46.

Submitted May 8, 2014

Accepted June 16, 2014

APPENDIX A

Sample (Spanish) Qualification Survey QuestionnaireQualification SurveyACTFL/UCLA NHLRC Heritage Speakers Oral Proficiency StudyThe ACTFL is seeking heritage speakers of Spanish to participate in a research project. Thepurpose of this study is to gather information about the linguistic profiles of heritage speakers.In order to qualify for this research project, you must be:

• A heritage speaker� of Spanish• 18 years or older• Willing to complete a qualification survey pertaining to education and languagebackground, preferences, and use

�For this study, the term heritage speaker is defined as an individual who:& Learned the heritage language (HL) in an informal setting (i.e., at home, in the

community) and uses the HL with family, friends, coworkers, etc.& Lives in the United States and has received the majority of his/her education in English‐speaking schools

Once qualified as a heritage speaker of Spanish, participants must agree to take an Internet‐delivered assessment of spoken language proficiency—the ACTFL OPIc—in the HL and toallow the use of the speech sample for research purposes. Participants will be paid $25� and

442 FALL 2014

will receive an official ACTFL OPIc certificate indicating their level of spoken proficiency inthe language according to the ACTFL Proficiency Guidelines–Speaking.

�Please note that the $25 compensation is only provided to those who qualify as heritagespeakers and agree to take the ACTFL OPIc.

Thank you for taking the time to complete this qualification survey.1. Do you speak Spanish?

a) yesb) no

If your answer to question 1 was yes, please continue to question 2.If your answer to question 1 was no, you may end the survey now.

2. Where/how did you first learn to speak Spanish? (Please check all that apply)a) at home with familyb) from your community (workplace, church, relatives, friends, etc.)c) in my native country where Spanish is spokend) living abroad (Peace Corps, missionary, etc.)e) other _______________________________________________

3. Which of the following best describes your speaking ability in Spanish?a) I can say only a few words and phrases. It may be difficult to understand what I say inSpanish.b) I can say enough to survive in a Spanish‐speaking environment (i.e., order a meal, buy atrain ticket, ask questions, deal with a simple social situation). A sympathetic listener will beable to understand what I say in Spanish.c) I can tell stories; explain situations; clarify miscommunications; and describe people,places, and things. I have enough language to make explanations even when there is anunexpected turn of events. Most native speakers of Spanish will understand what I say when Ispeak Spanish.d) I can support opinions, deal with abstract issues, and speak hypothetically with virtually noerrors in language.e) My ability in Spanish is equal to that of a highly articulate, educated native speaker. I amable to tailor my language to all audiences and speak with subtlety and nuance.4. Are you willing to participate in a compensated research study on your HL?a) yesb) no5. Are you willing to complete a background survey about how you learned and use yourlanguages? It will take approximately 3 to 5 minutes.6. Are you willing to take an Internet‐delivered test of spoken ability in your HL? It will takeapproximately 30 minutes.7. Are you willing to have the sample of language used for research purposes?If your answers to questions 4, 5, 6, and 7 are all yes, please provide us with your contactinformation (all information will remain confidential and will be used strictly for researchpurposes).

Foreign Language Annals � VOL. 47, NO. 3 443

APPENDIX B

Sample (Russian) Background Survey QuestionnaireBackground SurveyACTFL/UCLA NHLRC Heritage Speakers Oral Proficiency StudyThe ACTFL is seeking heritage speakers of Russian to participate in a research project. Thepurpose of this study is to gather information about the linguistic profiles of heritage speakers.In order to qualify for this research project, you must be:

• A heritage speaker� of Russian• 18 years or older• Willing to complete a qualification survey pertaining to education and languagebackground, preferences, and use

�For this study, the term heritage speaker is defined as an individual who:• Learned the heritage language (HL) in an informal setting (i.e., at home, in the community,or in the country of the language) and uses the HL with family, friends, coworkers, etc.

• Lives in the United States and has received the majority of his/her education in English‐speaking schools

Once qualified as a heritage speaker of Russian, participants must agree to take an Internet‐delivered assessment of spoken language proficiency—the ACTFL OPIc—in the HL and toallow the use of the speech sample for research purposes. Participants will be paid $25� andwill receive an official ACTFL OPIc certificate indicating their level of spoken proficiency inthe language according to the ACTFL Proficiency Guidelines–Speaking.

�Please note that the $25 compensation is only provided to those who qualify as heritagespeakers and agree to take the ACTFL OPIc.

Thank you for taking the time to complete this qualification survey.Please provide us with your contact information (all information will remain confidential andwill be used strictly for research purposes).1. Do you speak Russian?a) yesb) no(If you answer no, you may stop the survey now.)2. What is your country of birth? _________________________________________3. If youwerenotborn in theUnited States, howoldwere youwhenyouarrived in theUnited States?a) younger than 2b) 2–5 years oldc) 6–10 years oldd) 11–13 years olde) 14–18 years oldf) older than 184. Did you live in another country other than the former Soviet Union before coming to theUnited States? If yes, for how long?a) less than 1 yearb) 1–2 yearsc) 3–5 yearsd) more than 5 years5. What is your age?a) 18–25b) 26–35

444 FALL 2014

c) 36–45d) 46–55e) 56þ6. Which statement below best describes your profession? (please circle all that apply)a) I am a student enrolled in high school.b) I am a student enrolled in a college or university.c) I work outside the home.d) I work from home.e) I do not study or work.f) Other _____________________________________________________7. What language do you speak most often?a) Russianb) Englishc) A mix of Russian and English8. What language do you speak most often at home?a) Russianb) Englishc) A mix of Russian and English9. What language do you speak most often with your friends?a) Russianb) Englishc) A mix of Russian and English10. In which language do you consider yourself most proficient?a) Russianb) Englishc) I am equally proficient in both English and Russian.11. Have you formally studied/taken classes in the United States in Russian?a) Yes, I attended classes in the United States.b) No, I have not attended classes in the United States.12. Have you had any formal schooling in the country where Russian is the native language?a) yesb) no13. Which statement best describes the course(s) you took in Russian? (please circle all thatapply)a) I studied Russian in high school.b) I studied Russian in college or at university.c) I studied Russian in a community language class.d) Other _____________________________________________________14. Do you read and write in Russian?a) yesb) no15. If you answered yes to question 14, did you learn to read and write in a country whereRussian is the native/second language?a) Yes.b) No, I learned to read and write in the United States.16. How often do you travel to the country where Russian is the native language?a) annuallyb) biannuallyc) three to five yearsd) five years or more

Foreign Language Annals � VOL. 47, NO. 3 445

e) never17. If you travel to your country where Russian is the native language, what do you do there?a) visit relativesb) visit as a touristc) attend classesd) other ________________________________________________18. Would you be interested in participating in a compensated research study on HLs? If yes,you may be contacted by phone or e‐mail with more information.a) yesb) no

APPENDIX C

OPIc Rater Review Form Summary

APPENDIX D

OPIc Heritage Speaker Linguistic Sample Review Form

Level of Sustained Performance Level of Linguistic Breakdown

Rating Criteria Evidence of BreakdownFunctions FunctionsContext/content Evidence of linguistic breakdownComprehensibility Text typeText type Comprehensibility

Specify other forms of breakdownQuantity and quality of performancePrimary deficiencies that prevent the speaker from functioning at the________ level are:Comments:

Rating Responses with

Full Control

Responses with Evidence

of Breakdown

Intermediate Intermediate AdvancedAdvanced Advanced SuperiorSuperior Superior

Quality of Sustained Performance at Level

Linguistic Features in Each Response Signs of Breakdown

Fluency Ability to address the taskFeatures of pronunciation Functional breakdownVocabulary Linguistic breakdownAccuracy: grammar and structurePragmatic competenceSociolinguistic competenceText type

Quantity and quality of performancePrimary deficiencies that prevent the speaker from functioning at the _____ level are:Comments:

446 FALL 2014