Back to AAIR Journal Volume 13, No. 1

Comparative Analysis of Student Surveys on International Student Experience in Higher Education

Published in Volume 13, No. 1, 01 November 2007

Printer friendly version

Leonid Grebennikov and Ivan Skaines, Office of Planning and Quality, University of Western Sydney

This paper was first presented at the Australasian Association for Institutional Research 2006 Forum on 20 October 2006.

Abstract

This article seeks to contribute to the current discourse on internationalisation of higher education in Australia and to understanding and managing the increasing diversity of the student body. Its specific concern is a comparison of international versus domestic student experience regarding a range of a university’s academic and general services provision. The research sample consisted of 5,090 students who participated in three University of Western Sydney (UWS) surveys in 2004–05. The results indicated that international and domestic student experience at UWS was largely similar in learning and teaching; however, overseas undergraduate students were significantly less satisfied with course outcomes, provision of facilities and resources — and access to them, than domestic students. No such differences were evident at postgraduate and research higher degree levels. One practical outcome of this study was a clear need for promotion and communication of various support services and facilities for international students, especially at the early stages of their program.

The number of overseas students enrolled in Australian higher education institutions grew from 40,000 in 1994 to 344,815 in 2005 (Australian Education International, 2005). Worldwide the numbers of students seeking education outside of their home country is expected to grow from 1.8 million students in 2000 to 7.2 million by 2025 (Asia New Zealand Foundation, 2003). The international students studying in Australia’s institutions are primarily from the high socioeconomic groups in Asia (Scott, 2005). They cannot but look at what constitutes the best value for money education across the world and some may reassess whether they might not be better to study, for example, in the USA, New Zealand, or at home.

There has been a dramatic fall in the proportion of federal funds allocated to Australian universities over the past decade (Currie, 2005; Scott, 2005; Soutar & Turner, 2002). This has increased the importance of full fee-paying overseas students as a source of income, and thus competition between export education providers to be responsive to international student needs. With this competition has come a significant increase in foreign students’ expectation that their total experience at university, not just what happens in the classroom, be of the highest quality (Scott, 2005). Because of the importance of international students to the Australian higher education system, it is essential to understand specifically what they want and need in order to gain and retain them.

Research in various universities throughout the world have shown that international students’ perceptions of quality are shaped by non-academic aspects of their university experience no less than by academic ones (e.g., Abadi, 2000; Al-Mubarak, 2000; Cameron & Meade, 2002; Eland, 2001; Hechanova-Alampay, Beehr, Christiansen, & Van Horn, 2002; Heggins & Jackson, 2003; Ramsay, 1999; Washburn, Beaumont, & Teasdale, 2001). For example, Washburn, Beaumont, and Teasdale (2001) from the University of British Columbia, Canada, found that obtaining clear information and timely feedback regarding the university while in their home country; assistance with accommodation upon arrival; cross-cultural training opportunities; Orientation and Peer Support programs; and availability of extensive and ongoing English tutorial sessions were rated most highly by international students. A qualitative research into satisfaction with the Oklahoma State University, USA, among international students (Abadi, 2000) reported that while 64% of respondents were satisfied with their academic experience at the university, only 42% and 36% expressed satisfaction with their personal and social experiences respectively. The respondents emphasised the importance of international student advisors; provision of activities for foreign students, such as home visits and host family programs; and access to the library and computer labs. Al-Mubarak (2000) investigated international student experience at the Pennsylvania University, USA, and came to the conclusion that most international students encountered adjustment problems in areas such as: communication, cultural, personal, health, housing, and food. Most international students (90%) experienced problems in the first year of their program. Students sought help from a friend, school office, classmate, teaching staff, family, or advisor (in order of preference) when they experienced problems. Cameron and Meade (2002), New Zealand researchers from the University of Otago, named promotion and communication of various support services and facilities among incoming international students as a key factor supporting their transition to university.

During the period from 1994 to 2005 the proportion of overseas students at UWS has grown from a few per cent to 19% of its total enrolment. Yet, in 2004–05 UWS experienced a deficit of international students intake, primarily because of its deliberate withdrawing from provision of offshore programs with low return on investment. In 2003, there were 8,455 overseas students enrolled at UWS, including 3,877 onshore and 4,578 offshore students. In 2005 the corresponding numbers dropped down to 5,448, 3,654 and 1,794. However, UWS is not alone in this regard. The number of international students applying to study in Australia dropped over the period 2003–05 in several universities. For example, the University of Newcastle and Monash University were reported to have experienced a drop in international student enrolments in 2005 of 9% and 15% respectively (Jopson & Burke, 2005).

In such a context, it is critically important for UWS to determine precisely what attracts and retains students with diverse cultural backgrounds. This was one of the main tasks of an extensive research agenda the university has undertaken since April 2004. This article provides a comparative analysis of three relevant surveys conducted in 2004–05. These were the UWS Student Satisfaction Survey (SSS) covering the total UWS experience; the UWS Research Student Satisfaction Survey (RSSS) completed by higher degree research (HDR) students; and the Course Experience Questionnaire (CEQ) completed by graduating undergraduate students.

The principal objective of this article is to summarise the benchmarked results of the surveys with respect to the international student experience at the university. Particular attention was focused on university’s services and areas of provision assessed differently by overseas and domestic students. Identification of such domains where differences were most evident could enabled the university to suitably customise them, which could help ensure quality learning and teaching for all students, and maintain the UWS student population.

Method

Participants and procedure

The UWS Student Satisfaction Survey 2004–05 was completed by 3,166 onshore coursework students via scannable form (1,695 records) or online (1,471 records). The majority of respondents (77%) were undergraduate students at the bachelor level. Due to technical reasons, only the data obtained via scannable forms were used in this research, excluding the records with residential status unspecified. The final data comprised 1,496 records of 1,161 domestic and 335 international students. The research sample was representative of the university’s profile in terms of gender, age, college, level of study, campus, Aboriginal or Torres Strait Islander descent, medical condition, payment mode and language background. The similarly representative Course Experience Questionnaire sample of 3,201 graduates who completed an award course in 2005, consisted of 2,819 domestic and 382 international students. It included 2,087 records of 1,870 local and 217 international students containing written responses to open-ended questions. These were used in the content analysis of comments via CEQuery — an IT-based qualitative analysis tool. The Research Student Satisfaction Survey was completed by a sample of 411 UWS HDR students in 2005. Of those respondents, 357 were local, 36 were international students, and 18 students did not specify their residential status, thus the final data of 393 records were analysed.

Instruments

The UWS Student Satisfaction Survey builds on a tool developed at the University of Technology, Sydney (UTS), and used at the Tshwane University of Technology, South Africa, and in a range of other universities. It allows the university to identify key areas of good practice, priority areas requiring enhancement and to undertake benchmarking for improvement with other institutions using the same survey. Students rate a set of 85 items designating the university’s academic and general services and facilities on their importance and performance, using a five-point Likert-style scale (1 = low to 5 = high). These items cover students’ total experience at UWS and include:

• Current Course Experience (e.g., unit content, materials, methods, class sizes, staff)
• Outcomes of Study (e.g., development of relevant abilities and skills)
• Administration and Access to facilities and resources
• Learning Support Services (e.g., library, WebCT, computing facilities)
• Student Services (e.g., Careers and Employment Advice, Counselling)
• General Student Facilities (e.g., food services, shops, sport and fitness)
• UWS Student Association’s services and activities.

The Course Experience Questionnaire consists of 24 items, plus one overall satisfaction item, and aims to measure students’ perceptions of various aspects of the course just completed. Graduates are asked to indicate the extent to which they agree or disagree with each of 25 statements using a five-point Likert-style scale where 1 represents strong disagreement and 5 strong agreement. The 24 items relate to five aspects of students’ experience of their degree course: Good Teaching, Clear Goals and Standards, Generic Skills, Appropriate Assessment and Student Support.

Respondents are also invited to answer two questions in their own words: ‘What were the best aspects of your course?’ and ‘What aspects of your course are most in need of improvement?’. The written comments are automatically classified by the CEQuery qualitative analysis tool into five main domains: Staff, Course Design, Outcomes, Assessment and Support, and 26 subdomains using a custom-tailored dictionary. The CEQ is based on decades of research and refinement, and is fully evaluated from the psychometric point of view (Graduate Careers Council of Australia, 2004, 2005).

The UWS Research Student Satisfaction survey was developed from a parallel instrument used over the past five years at UTS. It was refined in consultation with key UWS staff, including the UWS Research Studies Committee. The survey was trialled with a range of UWS HDR students and further refined in light of their feedback. The survey’s design and structure are very similar to those of the SSS. Its 107 items cover areas of HDR students’ experience such as:

• Academic Support
• Outcomes of Study
• Administration and Access to facilities and resources
• Library Services
• Computing Facilities, Research Support and Infrastructure
• Student Services
• Minimum Resource Policy
• General Student Facilities
• UWS Postgraduate Student Association’s services and activities.

Results

To determine whether differences between participants’ mean ratings of survey items were evident on the basis of residency, an independent t test was conducted for each survey data with residential status as the independent variable. Note that the t test is relatively robust with respect to departures from normality, provided that sample sizes are large enough. In this study all sample sizes are sufficiently large to justify the use of the t test. The level of significance was set at p < .01 for the SSS analysis in order to select the most important differences out of many. As the analyses of the CEQ and RSSS showed fewer differences between mean scores of local and international students, a less strict significance level of p < .05 was applied. Due to large numbers or items per survey only significant results are presented in Tables 1–6.

UWS Student Satisfaction Survey

Table 1 shows that domestic UWS students rated the importance of half of the Course Experience items and all but two Administration items higher than international students. International students rated the importance of Access to sport and religious facilities, and to public transport; English Language Intensive Courses; International Student Advisors and Office; Student Association providing collectives, clubs and societies; and the importance of almost all General Facilities higher than local students.

Table 1: Results of Independent Samples Test Comparing SSS Mean Ratings on Importance by Residential Status*

Table 2 shows that domestic students were satisfied with teaching staff and methods; many course outcomes, learning support and general facilities significantly higher than international students. International students were not satisfied with any services higher than domestic students.

Table 2: Results of Independent Samples Test Comparing SSS Mean Ratings on Performance by Residential Status*

The services rated significantly higher on importance and lower on performance by international students, compared to domestic students, represented areas which, from the undergraduate overseas students’ perspective, needed priority attention and enhancement.

These were:

• Access to sport facilities
• Access to public transport
• Provision of shops and newsagencies
• Provision of sports and fitness facilities
• Provision of sport activities and programs
• Provision of swimming pools.

A similar pattern of standard deviation values for both domestic and international students can be seen in Tables 1 and 2. Moderate standard deviations emerged for academic and administrative items (mean ratings ranged from .61 to 1.13), and relatively higher ones for access, support, and general facilities items mean ratings (range = .45–1.62).

Course Experience Questionnaire

As may be seen in Table 3, of 25 CEQ items the mean scores on six items appeared significantly different on the basis of residential status (p < .05). International graduates had higher means on two items, both from the Good Teaching (GT) scale:

GT: The teaching staff on this course motivated me to do my best work;
GT: The staff made a real effort to understand difficulties I might be having with my work.

Local graduates rated higher all three items of the Appropriate Assessment (AA) scale (all reversed scored as they are negatively phrased), and also one item of the Generic Skills (GS) scale:

AA: To do well in this course all you really needed was a good memory;
AA: The staff seemed more interested in testing what I had memorised that what I had understood;
AA: Too many staff asked me questions just about facts;
GS: The course improved my skills in written communication.

No significant differences across residential status were found for the Clear Goals and Student Support scales.

Table 3: Results of Independent Samples Test Comparing CEQ Mean Ratings by Residential Status*

The CEQ’s open-ended comments classified using CEQuery were scored, and an independent t test was conducted with residential status as the independent variable to examine the differences between the mean scores on the CEQuery domains and subdomains. These means were calculated based on -1 = Needs Improvement, 0 = No Comment, 1 = Best Aspect. Thus, mean values > 0 denote more positive than negative comments, mean values < 0 denote more Journal of Institutional Research 13(1), 97–116. 108 negative than positive comments, mean values close to 0 indicate balanced or patchy opinions. The records without any comments were not analysed, as were those with comments unspecified by CEQuery. Four significant results of this analysis, as presented in Table 4, were:

• Course Design domain attracted slightly more positive than negative comments fromlocal students, and more negative than positive comments from international students;
• Assessment domain received more negative than positive comments from both subsamples, but with significantly larger number of such comments from local students;
• Intellectual Outcomes subdomain achieved more positive than negative comments from both subsamples, but with significantly more such comments from international students;
• Student Administration subdomain attracted more negative than positive comments from both subsamples, but with significantly larger number of such comments from local students.

Two results on CEQuery subdomains approaching significance were also included in Table 4 in order to better understand the significant results on the Course Design and Assessment domains.

Table 4: Results of Independent Samples Test Comparing CEQ Comment Mean Scores by Residential Status*

UWS Research Student Satisfaction Survey

Only four significant differences between international and local HDR students’ scores on importance were found at p < .01. As shown in Table 5, items which international HDR students rated significantly higher than domestic students were:

• Access to computers
• Access to public transport
• International Student Advisors
• International Student Office.

Table 5: Results of Independent Samples Test Comparing RSSS Mean Ratings on Importance by Residential Status*

With the p-level set at p <.05, four more items in the importance category emerged as significantly different, with all mean ratings higher for international students. These items were:

• Supervisors available when needed
• The annual progress reports help to monitor progress effectively
• Religious facilities
• Social activities.

In the performance category (Table 6) no significant differences were evident at p < .01, and four such differences emerged at p < .05, again all mean ratings were higher for international students. These items were:

• Knowledge of intellectual property issues such as copyright, patents, confidentiality
• Availability of computers for loan
• International Student Advisors
• International Student Office.

Table 6: Results of Independent Samples Test Comparing RSSS Mean Ratings on Performance by Residential Status*

The important outcome of this analysis was that there were no services that were rated by international HDR students significantly higher on importance, and at the same time lower on performance compared to domestic students. This suggests that international HDR students were satisfied with the services they consider important, similar or higher than local HDR students.

Discussion

Looking at the numbers of significantly different mean ratings per survey on a proportional basis, it appears reasonable to assume that the more mature students are (from undergraduates to graduates and HDR), the less differently local and international students perceive the university’s services and areas of provision in terms of both importance and performance. This assumption seems valid, even considering less sample size engaged in the RSSS than in the other surveys.

• SSS: 36 of 85 (42.4%) item ratings different at p < .01 on importance and 27 (31.8%) on performance
• CEQ: 6 of 25 (24.0%) item ratings different at p < .05
• CEQuery: 4 of 31 (12.9%) variable means different at p < .05
• RSSS: 8 of 107 (7.5%) item ratings different at p < .05 on importance and 4 (3.7%) on performance.

The results of the SSS indicate that international undergraduate students rate the importance of a number of academic and administrative areas, and the performance of many general facilities, access and support areas, as well as many course outcomes significantly lower than Australian residents. Further, no items on performance are rated higher by international students, while they rate significantly higher the importance of several access items and support services, and almost all general facilities. Services rated significantly higher on importance and lower on performance by international students compared to domestic students — such as access to public transport or sport facilities — represent areas for high priority attention and potential improvement.

These results are consistent with the outcomes of similar studies in Canada (Washburn, Beaumont, & Teasdale, 2001), New Zealand (Cameron & Meade, 2002), and the USA (Abadi, 2000). Therefore, it might be argued that the results reflect a broader issue for the export education industry across the world. As transition to and adjustment at the university are much more difficult for international students than for local residents, it may be a matter of vital importance for overseas students to feel welcome, cared for and become integrated into the student community. Thus, international students may be very demanding concerning provision, quality and accessibility of social activities, clubs, societies, shops, cafeterias, sport, religious, and other student facilities. They may also worry less about academic issues and administration services initially, and are less sure about course outcomes at the early stage of their program than are local students. Other SSS results are selfexplanatory; for example, public transport rated significantly higher on importance and lower on performance by international students, and English Language Intensive Courses, International Student Advisors, and International Student Office rated much higher on importance by international students.

It is interesting to note the similar pattern of standard deviation results in Table 1 and Table 2. For both domestic and international students the standard deviation values are moderate for the mean ratings on Course Experience, Outcomes and Administration, and commonly higher for the mean ratings on Access, Learning Support and General facilities. In view of the multi-campus nature of the University ofWestern Sydney, this pattern may suggest that while the standards of teaching, learning and administration are equivalent throughout the university, the availability and quality of facilities, resources and support services may vary across campuses. This most likely caused large score variation for all students, regardless of their residency.

Unexpectedly, one outcome of the SSS disagrees with two outcomes of the CEQ. An SSS item in the performance category rated significantly lower by international students reads: ‘The course is conducted by staff who are good teachers’. Two staff-related CEQ items rated significantly higher by international graduates are: ‘The teaching staff on this course motivated me to do my best work’, and ‘The staff made a real effort to understand difficulties I might be having with my work’. This inconsistency could occur due to, first, more generic and impersonal phrasing of the SSS item and more specific and personal phrasing of the CEQ items. The words ‘motivated’, ‘to do my best’, and ‘understand difficulties’ might trigger recollection of staff members particularly responsive to student needs. Second, the graduates’ assessment may be more mature as they met more staff and had more opportunities to understand their conduct by the end of their program. This may especially refer to international students who are sensitive to help and motivation provided by academic staff.

Three CEQ items in the Appropriate Assessment scale, all focused on memory-based assessment, are rated lower by international students —meaning they agree more strongly than domestic students that assessment was too much memory focused. This may be explained by their different English language skills, so that international students generally had to memorise more material than local students and that this might impact on their perception of academic assessment. One more CEQ item rated significantly lower by overseas students is ‘The course improved my skills in written communication’ of the Generic Skills scale. This result seems unexpected, since if a course is really helpful in terms of such skills, international students should be the ones to perceive and value this more than local residents. The only reasonable interpretation of this outcome is that international students may be too severe on themselves regarding English language and do not take notice of moderate improvements, while local students may do.

The CEQuery results suggest that international students are less satisfied with course design than local students. This may refer to the practical–theory links subdomain, as this is the only course design subdomain that shows difference between the subsamples approaching significance (t(2,085) = -1.88, p = .06). The comments allocated to this subdomain are about sufficiency of vocationally relevant components in the course design and balance of theory and practical subjects.

Both subsamples are dissatisfied with assessment but local students more so than international — in particular, this difference may refer to the expectations subdomain (t(2,085) = 1.94, p = .052). This subdomain comprises comments on clarity and consistency of assessment tasks, guidelines, submission deadlines and grading criteria. Both groups are contented with intellectual outcomes, but international students more so than local. The apparent conflict between the CEQuery and SSS scores, suggesting that overseas students are less satisfied with their course outcomes, may have the same origin as the already mentioned conflict between the CEQ and SSS results. This is that undergraduate international students may be less sure about course outcomes at the early stage of their program than are local students, but may value these outcomes equally or even higher than domestic students at the end of their program. Finally, both groups are critical regarding administration with local students making more disapproving comments than international students. Thus, the only area international graduating students made significantly less positive and more negative comments, compared to domestic graduates, is course design.

The RSSS outcomes suggest equal level of satisfaction of international compared to domestic HDR students with the university’s academic and general service provision. Of 107 items compared, only four reached significance on performance and all were rated higher by overseas HDR students. Importantly, of eight items they rate higher on importance, none are rated lower on performance, and two services — International Student Advisors and Office — significantly higher on performance. These services represent areas of particularly good practice from overseas HDR students’ perspective.

In summary, the main message from this analysis is that international student experience at UWS is largely similar to domestic student experience in learning and teaching; however, there are differences in satisfaction with provision, quality and accessibility of facilities and resources. Overseas undergraduate students have rated higher than domestic students the importance of many of the university’s facilities, while the performance of and access to some of these facilities have been rated lower by international compared to local students. The great care overseas undergraduate students devote to social activities, clubs, societies, shops, cafeterias, sport and religious facilities, and public transport is quite understandable given that these services are most likely to be used by students studying away from home. Therefore, it is important to ensure that these facilities are of high quality standard, affordable, easily accessible, regularly monitored and equivalent across the UWS campuses. As no difference in satisfaction on the basis of residency has been evident at postgraduate and HDR levels, one more outcome of this research is a clear need for promotion and communication of various facilities and support services available for international students, especially at the early stages of their program.

References

Abadi, J. M. (2000). Satisfaction with Oklahoma State University among selected groups of international students. Dissertation Abstracts International, 60(8-A), 2821.

Al-Mubarak, A. H. (2000). Adjustment problems, coping methods, and choice of helpers of international students attending a large Pennsylvania university. Dissertation Abstracts International, 61(1-A), 94.

Asia New Zealand Foundation. (2003). The Export Education Industry: Challenges for New Zealand. Occasional Paper. Retrieved March 6, 2006, from http://www.asianz.org.nz/research/research-internationaleducation.php

Australian Education International. (2005). Recent Annual Statistics. Department of Education, Science and Training, Australian Government, Canberra. Retrieved March 6, 2006, from http://aei.dest.gov.au/AEI/MIP/Statistics/StudentEnrolmentAndVisaStatistics/Recent.htm

Cameron, B., & Meade, P. H. (2002). Supporting the transition to university of international students: Issues and challenges. Proceedings of the 6th Pacific Rim Conference on First Year Experience in Higher Education. Christchurch, 8–10 July, (CD).

Currie, J. (2005). Organisational culture of Australian universities: Community or corporate? Proceedings of the 2005 HERDSA Annual Conference. Sydney, Australia 3–6 July, Vol. 28, 5–16.

Eland, A. J. (2001). Intersection of academics and culture: The academic experience of international graduate students. Dissertation Abstracts International, 62(3-A), 909.

Graduate Careers Council of Australia. (2004). Course Experience Questionnaire 2003. (ISSN Publication No. 1325–7528). Melbourne, Australia: GFS Printing Services Pty. Ltd.

Graduate Careers Council of Australia. (2005). Graduate course experience 2004: The report of the Course Experience Questionnaire. Melbourne, Australia: GFS Printing Services Pty. Ltd.

Hechanova-Alampay, R., Beehr, T. A., Christiansen, N. D., & Van Horn, R. K. (2002). Adjustment and strain among domestic and international student sojourners: A longitudinal study. School Psychology International, 23(4), 458–474.

Heggins, W. J. III, & Jackson, J. F. (2003). Understanding the collegiate experience for Asian international students at a Midwestern Research University. College Student Journal, 37(3), 379–391.

Jopson, D., & Burke, K. (2005, May 7). Campus critical: The privatisation of learning has come at a huge cost. Sydney Morning Herald, p.1.

Ramsay, S. (1999). Academic adjustment and learning process: A comparison of international and local students in first year university. Higher Education Research and Development, 18(1), 129–144.

Scott, G. (2005). Promoting student retention and productive learning in universities: Research and action at UWS 2004–05. Unpublished manuscript, Office of Planning and Quality, University of Western Sydney, Penrith, Australia.

Soutar, G. N., & Turner, J. P. (2002). Students’ preferences for university: A conjoint analysis. The International Journal of Educational Management, 16(1), 40–45.

Washburn, C., Beaumont, K., & Teasdale, J. (2001). Internationalizing student success: Focus group research project [University of British Columbia, Planning and Institutional Research website]. Retrieved March 9, 2006, from http://www.pair.ubc.ca/studies/isi01.htm

Correspondence to:

Leonid Grebennikov
Office of Planning and Quality
University of Western Sydney
PO Box 1000
St Marys
NSW 1790 Australia

E: L.Grebennikov@uws.edu.au