Michael J. Meaney
Ph.D. Candidate, Economics and Sociology of Education
Gates Cambridge Scholar
University of Cambridge
Visiting Research Fellow
Action Lab at EdPlus
Arizona State University
Funding for this work provided, in part, by the Gates Cambridge Trust. Research support provided by the Faculty of Education at the University of Cambridge, and the Action Lab at Arizona State University.
CLICK TO READ PAPER – CLICK TO DOWNLOAD PAPER
This paper discusses the rapid skills-biased technological transformation of the labor market, as well as the ever-widening global supply-demand gap for higher education. Given that the provision of higher education around the globe, all but required to secure higher wage premiums in the information economy, is heavily stratified along racial and socioeconomic lines, these trends threaten to exacerbate already historic levels of inequality. MOOCs, heralded as a potential antidote to this problem, are actually exacerbating this problem, as a result of what I call hegemonic design bias. This bias stems from three sources: 1) most MOOCs are developed by prestigious universities that derive their prestige based on whom they exclude, rather than include, and this pattern seems to be replicated in the course content and teaching style of their MOOCs; 2) stale and retrograde pedagogy predicated on the behaviorist paradigm of knowledge transfer rather than knowledge co-creation; and 3) an iteration inequity loop, whereby data from existing courses, at present taken primarily by college educated users (~80% of users), is mined and used to optimize future design, tailoring MOOCs to fit the behavior patterns of those less likely to need help.
Skills-Biased Technology Change And The Future Of Employment
The diffusion of technology into every aspect of modern life is radically altering the nature of the economy and the ways in which humans contribute to economic output. As the price per processing unit power of computers continues to fall exponentially, and an artificial intelligence revolution begins to take place, the nature of the labor market and the skills required to succeed within it will continue to shift at an ever-increasing pace (Lewis-Kraus 2016).
This is detailed in a 2013 seminal study by Michael Osborne at the University of Oxford, which predicted that 47 percent of jobs in the United States alone were at risk of automation. Job dislocation will force workers on the lower end of the skill spectrum to acquire new skills in order to find employment. Automation will principally affect “…low-skill and low-wage occupations.” The study implied that “… as technology races ahead, low-skill workers will reallocate to tasks that are non-susceptible to computerization – i.e., tasks requiring creative and social intelligence. For workers to win the race, however, they will have to acquire creative and social skills,” (pg. 269).
A report by the McKinsey Global Institute examining the labor markets of 46 countries concluded similarly, adding a slightly optimistic hedge. “While about half of all work activities globally have the technical potential to be automated by adapting currently demonstrated technologies, the proportion of work actually displaced by 2030 will likely be lower, because of technical, economic, and social factors that affect adoption,” (Manyika et al. 2017, pg. 4).
The bottom line is that workers will be required to continuously upgrade skills in order to maintain employability in the labor-market, especially workers on the lower end of the skills spectrum (Autor 2014). Given that the present provision of education throughout developed and developing countries reinforces inequities along socioeconomic and racial lines, skills-biased technological change in the labor market threatens to exacerbate existing economic stratification and social inequality (for the American perspective, see Carnevale and Strohl 2013; for the global perspective, see Altbach et al. 2009).
Massive Open Online Courses (MOOCs) were heralded as a disruptive technological force that could help solve these problems by improving access to education for traditionally underrepresented students around the world (Selingo 2014). Many MOOCs provided classes from Harvard, Stanford, and the Massachusetts Institute of Technology. While the brick-and-mortar versions of these schools remain accessible to only a select few, it was hypothesized that MOOCs could perhaps enable anyone, anywhere, to receive a world class education, for free (Friedman 2013).
This hypothesis proved wrong, and demonstrably so. Moreover, MOOCs may be insidiously widening educational inequity gaps and economic disparities (Liyanagunawardena et al. 2014), a problem caused by how MOOCs are designed and compounded by how MOOCs are currently researched and developed.
This paper posits that MOOCs are infected by a hegemonic design bias. I begin with a review of the literature about global demand for higher education and who MOOCs are presently serving. From there, I will derive the notion of hegemonic design bias. I will then hypothesize the sources of this hegemonic design bias, borrowing insights from the sociology of technology and organizational theory. Then, I will consider design paradigms that may serve as an antidote to hegemonic design bias and some practical steps for how to incorporate this kind of thinking, as well as some explicit pedagogical interventions that might help MOOCs become more inclusive and equitable. To conclude, I will reflect on why it is essential to iterate upon and improve MOOCs in order to ameliorate worsening levels of inequality.
Global Demand For Higher Education, And Who MOOCs Serve
As the international economy becomes increasingly interdependent, demand for higher education has surged. The forces of globalization – the processes of economic integration, advancing technological development, and the emergence of an “international knowledge network,” – compel students and families to prioritize higher educational attainment as a necessity for social mobility and economic security (Altbach et al. 2009). Governments are charged with nurturing education systems that enable this dream to be realized, both in order to fulfil the expectations of their constituencies and as a way to maintain geostrategic competitive advantages in the globalized economy (Carnoy 2016). As a result of these trends, the enrolment ratio for higher education students globally – the proportion of student-age population attending higher education – doubled between 1992 and 2012 (Economist 2015). A recent UNESCO report estimated that 165 million people were enrolled in higher education (2013).
Given that roughly 30 percent of the world’s population is under the age of 15, projections suggest that by 2025 participation in higher education will reach 263 million. The delta between current supply of higher education infrastructure and imminent student demand is a gap of close to 100 million students (UNESCO 2013). India alone will account for some 140 million higher education-aged students by 2030, and presently can only accommodate about one-third of them with existing physical infrastructure (Kim 2015).
Developed nations are also struggling to satisfactorily educate their populations to keep pace with the talent needs of an increasingly complex knowledge-based economy. The U.S. will likely face a shortfall of 12 million higher education graduates in the next two decades (Carnevale and Rose 2015). These numbers belie a more insidious truth. The imbalance in supply and demand disproportionately affects traditionally underrepresented students. Asymmetries of information and inadequate access to resources can make the higher education process difficult to manage successfully. Increasing competitiveness drives admission rates downward in an attempt to drive prestige upward, oftentimes leaving societies’ most vulnerable students, those who stand to gain the most from higher education, the most at risk of never receiving one. Compounded educational deficits beginning in early childhood and primary school make it all the more difficult for students from disadvantaged backgrounds to thrive in secondary school and higher education (Reardon 2011).
Given this context, it is no wonder why MOOCs were met with such fanfare. MOOCs are “massive,” in that they can accommodate an unlimited number of students. MOOCs are “open” in that they require no application nor have any similar barrier to entry. MOOCs are fully completed “online.” And MOOCs are “courses” in the traditional sense, meaning that they cover a discrete range of content in a particular subject area, the completion of which indicates some form of mastery (Hollands and Tirthali 2014).
It is intuitive to see the appeal of this technology given the scope of the supply and demand problem, and the inequity problem. As the knowledge economy requires more and more learning to secure economic stability, could MOOCs play a role in helping reduce educational and economic inequality? The answer to this question depended on whether MOOCs were able to effectively serve traditionally underrepresented users.
Unfortunately, the empirical data suggests otherwise. A 2018 paper by van de Oudeweetering and Agirdag is a first of its kind systematic review of MOOCs and their implications for social mobility, centering on the question of whether MOOCs have been effectively reaching the traditionally underrepresented. Their findings show that, of the more than 400,000 MOOC users included in the studies they cover, nearly 80 percent already held a college degree. This finding has been replicated across nearly every MOOC study to date, a selection of which are included in the graphic below (data cited from: Robinson et al. 2015; Dillahunt et al. 2015; Christensen et al. 2013; van de Oudeweetering and Agirdag 2018; Ho et al. 2015; Wang et al. 2018).
Hegemonic Design Bias
How might we account for the relative lack of success? I hypothesize that MOOCs are a failure of design, stemming from three different components of their design production process. These components are: one, the source of MOOCs, the universities actually producing them; two, the actual design of the MOOCs themselves, the user experience and pedagogical techniques within them; and three, the research and development community around MOOCs. These failures contribute to an emergent hegemonic design bias inherent in MOOCs.
First, the majority of MOOCs are produced by elite institutions of higher education in the United States. These institutions, while often times preaching the virtues of access, equity, and inclusion, have been documented as some of the greatest contributors to the reproduction of socioeconomic inequity and inequality (Turner 2017). These universities have global reputations to further confirm and extend, and are staffed by faculty and academic administrators who are hyper-responsive to the cult of prestige in the academy, and the institutional isomorphism that results (Crow and Dabars 2015). These organizational structures and incentives prevent these institutions from producing the kind of content that would be most useful to scale to make the provision of higher education more equitable and inclusive, courses on remedial Algebra or Composition 101 for English Language Learners, for example. These kinds of courses would dilute the brands of these prestigious institutions and could attract students who face learning challenges these universities are ill-equipped to deal with.
Second, MOOCs suffer from a user experience and pedagogical problem. MOOCs are not particularly innovative in terms of pedagogy. MOOCs involve a sequenced delivery of content in a structured manner, and there is little emphasis on participants co-creating knowledge with one another. Distance learning expert Tony Bates describes MOOCs as stuck in a “very old and outdated behaviorist pedagogy, relying primarily on information transmission, computer marked assignments and peer assessment.” (2012). For students without a college degree, this is the sort of educational experience that might be psychologically overwhelming and threatening. Additionally, underrepresented learners could face pre-existing knowledge barriers that may contribute to feelings of panic or inadequacy that may predicate dropout (Oudeweetering and Agirdag 2017). The behaviorist paradigm is not conducive of reducing these challenges for students, and may actually make these challenges more acute.
The third and most vexing component of hegemonic design bias is an area unexplored by the research literature: the early-adopter bias of technology iteration (Meaney and Fikes 2018). This insight draws on concepts from sociology and organizational theory: the politics of technology and the diffusion of innovations.
The politics of technology is a concept articulated by Langdon Winner which claims that technologies have an inherent politics that influences who benefits from them (1980). He suggests that the design of technology “becomes a way of settling an issue,” meaning that the design dictates implications for usage (pg. 123).
The diffusion of innovation is a concept developed by Everett Rogers. The theory suggests that innovations diffuse across society along different segments of the population, sequentially: innovators, early adopters, early majority, late majority, and laggards (2010).
The politics of technology combines with the diffusion of innovations in a problematic way. Rogers notes that early adopters of new technologies will more likely be well-educated and wealthier. These users have access to more and better information, coupled with a higher tolerance of risk for new products (Rogers 2010). Early adopters are also likely to have disposable income and are a more attractive target market toward which to design new products. Innovations are optimized based on data available from early adopters.
This pattern of optimizing new products based on data from early adopters is the exact pattern that the MOOCs research and development community has followed. Learning analytics of massive data sets from MOOCs have focused on behavior patterns of average, early-adopting MOOC users. These users are much more likely to be already well-educated. This leads optimization and design recommendations to be driven by insights derived from users less likely to need help. If future MOOC iterations continue to be optimized based on present usage patterns of early adopters, and if these usage patterns continue to reflect the needs and behaviors of the already well-educated, this could further exacerbate educational inequity and economic inequality. This process is modeled in the figure below.
Thousands of papers have been published analyzing user behavior in MOOCs. Many of these papers evaluate MOOC persistence and completion as dependent variables of interest, and consider a number of course design features (length of video, scheduling of tests, language in emails, etc.) as well as a number of student engagement activities (log data of video watching, paper downloads, participation in forums, etc.), as independent variables of interest.
For example, a 2016 paper by Evans, Baker, and Dee analyzes over 2.1 million observations from 44 MOOCs offered through Coursera. They find that while length of video in MOOCs does not matter, the length of the course does (longer courses are associated with more attrition). Furthermore, they find that the optimal release pattern of videos in courses is in batches of two videos per week. At the student level, they find that students who enroll just before the course begins are more likely to persist then those who enroll long before or well after the course begins. Finally, the strongest predictor of course completion is whether a student completed the pre-course survey or not (Evans et al. 2016).
These are certainly interesting findings, and important. They do not however differentiate between behaviors among types of students based on educational attainment level, socioeconomic status, or any other categorical variable that might serve as a proxy for relative educational disadvantage.
A Way Forward
There are two design paradigms that might be considered when creating MOOCs in the future, if the aim is to have designs be more equitable and inclusive, which could allow MOOCs to potentially develop into a real vehicle for social mobility.
One paradigm comes from Batya Friedman of Colby College, who developed the theory of value centric design. Value centric design is a “theoretically grounded approach to the design of technology that accounts for human values in a principled and comprehensive manner throughout the design process” (pg. 1, 2013). Friedman discusses the idea of “preexisting bias” that exists independently of the technology, but informs how the technology is developed. “Preexisting biases may originate in society at large, in subcultures, or in formal or informal organizations and institutions…This type of bias can enter a technology either through the explicit and conscious efforts of individuals or institutions, or implicitly and unconsciously, even despite the best of intentions,” (Friedman 1996).
Another design paradigm that might be considered is that of reflexive design, proposed by Phoebe Sengers from Cornell, along with other colleagues (2005). These researchers call for the development of technology to take a much more explicit approach in accounting for preexisting bias. Reflexive design would entail “reflection on unconscious values embedded in computing and the practices that it supports.” Furthermore, “reflective design…combines analysis of the ways in which technologies reflect and perpetuate unconscious cultural assumptions, with design, building, and evaluation of new computing devices that reflect alternative possibilities,” (pg. 49, 2005).
These paradigms would yield MOOC designs far more responsive to underrepresented students using MOOCs. It would force universities, technologists, professors, and designers, to consider specific difficulties and burdens that underrepresented learners might face when engaged with online learning.
Practically speaking, this would require two explicit steps. First, teams could clearly articulate the goals of making their MOOCs. Is the goal to serve traditionally underrepresented students and democratize learning for more people, or is it to provide content that requires some (reasonably high) minimum knowledge-level and cognitive capacity such that only certain learners would be able to engage? If the goal is to democratize learning, teams might reflect further on the following questions during the design planning process:
- Is the content produced reflective of the literacy and numeracy of the average non-college educated student?
- Are the pedagogical techniques scaffolded and differentiated so that different types of learners are able to effectively engage the material?
- What types of learners are the advertising and marketing campaigns targeting?
Second, teams could hypothesize the sources of their own pre-existing bias (educational, demographic, gender, socioeconomic, racial, cultural, cognitive) and determine intentional ways to mitigate it. Members of MOOC producing teams might ask themselves:
- How are my own personal experiences and judgements about the world informing how I am creating this content?
- Am I sure that someone with an opposite set of life experiences, or merely different ones, from me would want content created in the same way?
- Is my own language, culture, race, gender, academic background, or socioeconomic status overly influencing how I am thinking about this design process?
- Is there someone who might have a different perspective than me that I can I ask to double-check my instincts?
Short of completely redefining the design paradigm approaches used by universities to produce MOOCs, there are discrete pedagogical interventions that might also be helpful.
Rene Kizilcec of Stanford University developed an intervention to address social identity threat, one potential source of difficulty for underrepresented learners in MOOCs. He tested the intervention on users from lesser developed countries (LDC) in a randomized controlled trial, and it was found to deliver a consistently positive effect on learning outcomes (Kizilcec et al. 2017). A similar study was conducted in China, focused on lower-class men in an English-language MOOC as an at-risk group. The intervention led to improvements in grades, persistence, and completion rates (Kizilcec et al. 2017).
Prof. Kizilcec and I are running a similar experiment with an adaptive learning math MOOC offered by Arizona State University. The MOOC begins by having users take an hour-long skills assessment. This is very useful for building an adaptive curriculum; however, it is a very threatening way to start a course. Thousands of users are observed to drop out when prompted to begin the skills assessment, or shortly after starting. Our intervention engages the learner just before the skills assessment. Learners read an affirming message and are asked to reflect on the importance of taking the course and their plan to complete it. This may seem simple. It is. But these sorts of minor, social psychological interventions have been demonstrated to improve student learning outcomes.
Sound the Alarm!
The original intents and aims of MOOCs were laudable, or at least were framed in such a way. There are millions of learners still in need of flexible education options, and MOOCs and other forms of online learning may be the best vehicle to meet this increasing demand for higher education. Furthermore, as the pace of technological change continues to speed up, more and more workers will find their existing skills out of date and will be required to acquire new skills. Global educational infrastructure is not capable of meeting this demand, so MOOCs could serve to fill this void.
Institutions of higher education have a special responsibility to shape technology to help improve the world and make it a more just and equitable place. Right now, they are doing the opposite by producing MOOCs that serve the already well-educated. Additionally, the vast majority of research and discourse around MOOCs remains stuck in a rose-shaded glasses mode; most universities and MOOC providers are either unaware or blatantly ignoring the data demonstrating MOOCs as a potential agent of inequality reproduction.
All hope is not lost. If proper design considerations are made in creating future MOOC content, these institutions of higher education could make a momentous contribution to society by truly democratizing learning. As MOOCs remain nascent, they are highly malleable and not entrenched in any specific way that would prevent a direction change. Whether MOOCs are able to shift in the right direction is now up to the universities producing them.
Altbach, P. G., Reisberg, L., & Rumbley, L. E. (2009). Trends in global higher education: Tracking an academic revolution.
Autor, D. (2014). Polanyi’s paradox and the shape of employment growth (Vol. 20485). Cambridge, MA: National Bureau of Economic Research.
Bates, Tony. “What’s right and what’s wrong about Coursera-style MOOCs.” (2012).
Carnevale, A. P., & Rose, S. J. (2015). The Economy Goes to College: The Hidden Promise of Higher Education in the Post-Industrial Service Economy. Georgetown University Center on Education and the Workforce.
Carnevale, A. P., & Strohl, J. (2013). Separate & Unequal How Higher Education Reinforces the Intergenerational Reproduction of White Racial Privilege.
Carnoy, M. (2016). Educational Policies in the Face of Globalization: Whither the Nation State?. Handbook of Global Education Policy, 27.
Christensen, G., Steinmetz, A., Alcorn, B., Bennett, A., Woods, D., & Emanuel, E. (2013). The MOOC phenomenon: Who takes massive open online courses and why?.
Crow, M. M., & Dabars, W. B. (2015). Designing the new American university. JHU Press.
Dillahunt, T. R., Wang, B. Z., & Teasley, S. (2014). Democratizing higher education: Exploring MOOC use among those who cannot afford a formal education. The International Review of Research in Open and Distributed Learning, 15(5).
The Economist. “The World is Going to University.” March 28th, 2015. http://www.economist.com/node/21647285/print
Evans, B. J., Baker, R. B., & Dee, T. S. (2016). Persistence patterns in massive open online courses (MOOCs). The Journal of Higher Education, 87(2), 206-242.
Friedman, B. (1996). Value-sensitive design. interactions, 3(6), 16-23.
Friedman, B., Kahn Jr, P. H., Borning, A., & Huldtgren, A. (2013). Value sensitive design and information systems. In Early engagement and new technologies: Opening up the laboratory (pp. 55-95). Springer Netherlands
Friedman, T. (2013). Revolution hits the universities. The New York Times, 26(1), 2013.
Frey, C. B., & Osborne, M. A. (2017). The future of employment: how susceptible are jobs to computerisation?. Technological Forecasting and Social Change, 114, 254-280.
Ho, A., Chuang, I., Reich, J., Coleman, C., Whitehill, J., Northcutt, C., … & Petersen, R. (2015). HarvardX and MITx: Two years of open online courses fall 2012-summer 2014.
Hollands, F. M., & Tirthali, D. (2014). MOOCs: Expectations and Reality. Full report. Online Submission.
Kim, Joshua. “Coursera, Apple, and the Future of Global Higher Education.” Inside Higher Education. June 9, 2015. https://www.insidehighered.com/blogs/technology-andlearning/coursera-apple-and-future-global-higher-education
Kizilcec, R. F., Saltarelli, A. J., Reich, J., & Cohen, G. L. (2017). Closing global achievement gaps in MOOCs. Science, 355(6322), 251-252.
Kizilcec, R. F., Davis, G. M., & Cohen, G. L. (2017, April). Towards Equal Opportunities in MOOCs: Affirmation Reduces Gender & Social-Class Achievement Gaps in China. In Proceedings of the Fourth (2017) ACM Conference on Learning@ Scale (pp. 121- 130). ACM.
Lewis-Kraus, Gideon. “The Great A.I. Awakening.” New York Times (2016).
Liyanagunawardena, T. R., Williams, S., & Adams, A. A. (2014). The impact and reach of MOOCs: a developing countries’ perspective. eLearning Papers, 38-46.
Manyika, J., Lund, S., Chui, M., Bughin, J., Woetzel, J., Batra, P., … & Sanghvi, S. (2017). Jobs Lost, Jobs Gained: workforce transitions in a time of automation. McKinsey Global Institute.
Marope, P. T. M., Wells, P. J., & Hazelkorn, E. (Eds.). (2013). Rankings and accountability in higher education: uses and misuses. Unesco.
Meaney, M. & Fikes, T. (2018). Early-adopter Iteration Bias. Working Paper. Action Lab at Arizona State University.
Reardon, S. F. (2011). The widening academic achievement gap between the rich and the poor: New evidence and possible explanations. Whither opportunity, 91-116.
Robinson, A. C., Kerski, J., Long, E. C., Luo, H., DiBiase, D., & Lee, A. (2015). Maps and the geospatial revolution: teaching a massive open online course (MOOC) in geography. Journal of Geography in Higher Education, 39(1), 65-82.
Rogers, Everett M. Diffusion of innovations. Simon and Schuster, 2010.
Selingo, J. J. (2014). Demystifying the MOOC. The New York Times, 29.
Turner, N., & Treasury, U. S. (2017). Mobility Report Cards: The Role of Colleges in Intergenerational Mobility.
Sengers, P., Boehner, K., David, S., & Kaye, J. J. (2005, August). Reflective design. In Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility (pp. 49-58). ACM.
van de Oudeweetering, K., & Agirdag, O. (2017). MOOCs as accelerators of social mobility? A systematic review. Educational Technology and Society.
Wang, E., Fikes, T. & Pettyjohn, P. (2018). Open Scale Courses: Exploring Access and Opportunity for Less-Educated learners. Working Paper. Action Lab at Arizona State University.
Winner, L. (1980). Do artifacts have politics?. Daedalus, 121-136.
 These numbers themselves are somewhat incomplete in that they do not deal with a selection bias resultant from specific users being more likely to take surveys. Dealing with this issue econometrically is beyond the scope of this paper, though it is an area covered in my dissertation.