Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=vece20 The Journal of Economic Education ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/vece20 The effects of game-based financial education: New survey evidence from lower-secondary school students in Finland Panu Kalmi & Jaana Rahko To cite this article: Panu Kalmi & Jaana Rahko (2022) The effects of game-based financial education: New survey evidence from lower-secondary school students in Finland, The Journal of Economic Education, 53:2, 109-125, DOI: 10.1080/00220485.2022.2038320 To link to this article: https://doi.org/10.1080/00220485.2022.2038320 © 2022 The Author(s). Published with license by Taylor & Francis Group, LLC. Published online: 21 Feb 2022. Submit your article to this journal Article views: 2397 View related articles View Crossmark data The Journal of economic educaTion 2022, Vol. 53, no. 2, 109–125 The effects of game-based financial education: New survey evidence from lower-secondary school students in Finland Panu Kalmi and Jaana Rahko School of accounting and finance, university of Vaasa, Vaasa, finland ABSTRACT The authors of this article studied the effects of game-based financial education approaches using a sample of lower-secondary school students in Finland. The sample consisted of 640 students from 42 schools in different areas of the country. The authors focused on three different game-based interventions using a pre- and post-intervention survey design. They compared the effects of the interven- tions (and their combinations) to a control group that received only traditional teaching. They found robust effects with respect to knowledge gained from game-based approaches. However, the effects on self-reported financial behav- iors were weak. Abbreviations: ATT: average treatment effect on the treated; EIO: Economic Information Office; ITT: intention to treat; PISA: Programme for International Student Assessment; PS: propensity score There has been a considerable debate on the effectiveness of financial education in general. Doubts have been raised on 1) whether financial education influences financial behavior and 2) whether financial education even influences financial knowledge (e.g., Fernandes, Lynch, and Netemeyer 2014; Frisancho 2020; Lusardi and Mitchell 2014; Mandell and Klein 2009; Skimmyhorn 2016; Walstad, Rebeck, and MacDonald 2010; Willis 2008). Kaiser and Menkhoff (2017) summarized the recent research in their meta-analysis, where they found robust evidence of financial education impacting financial knowledge. They also found support for the proposition that financial education influenced behavior, although the effects were small and often focused on certain types of behaviors. These results were also supported by further analysis in Kaiser et al. (In press). According to Amagir et al. (2018), typical ways in which increases in financial literacy are assessed in a school context are 1) increases in knowledge, 2) changes in (self-reported) financial behavior, and 3) changes in attitudes. In assessing knowledge gains, studies in the secondary school context typically find improvements after educational interventions. The majority of studies also report changes in atti- tudes, whereas findings relating to behavioral changes are scarce. In a recent article, Kaiser and Menkhoff (2020) included even more recent studies but limited their focus on financial education in schools (pri- mary, middle, and high schools). They found very similar results to those in their previous meta-analysis in which all types of financial education produced relatively large effects on financial knowledge but small (although still significant) effects on behavior. Both Amagir et al. (2018) and Kaiser and Menkhoff (2020) speculated that the small effects related to behavior might be linked to the fact that schoolchildren rarely make any important economic decisions independently. While the focus of discussions initially has been largely on whether financial education has an impact, the discussion is gradually moving to the question of what actually works in financial edu- cation. Such studies include Drexler, Fischer, and Schoar (2014), who study rules of thumb vs. © 2022 The author(s). Published with license by Taylor & francis Group, llc. CONTACT Panu Kalmi panu.kalmi@uwasa.fi School of accounting and finance, university of Vaasa, Wolffintie 34, Vaasa, finland. https://doi.org/10.1080/00220485.2022.2038320 KEYWORDS Schools; economic education; financial education; financial literacy; games; Finland JEL CODES A21; G53; I21 RESEARCH IN ECONOMIC EDUCATION This is an open access article distributed under the terms of the creative commons attribution license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. 110 P. KALMI AND J. RAHKO principles-based approaches; Lusardi et al. (2017), who investigated the use of visual tools and nar- ratives in financial education; Kaiser and Menkhoff (2018), who found that active learning approaches worked better than traditional lectures in a sample of entrepreneurs; and Carpena et al. (2019), who studied the effects of personalized counseling and goal-setting. However, none of these studies took place in a school setting. In contrast, the recent article by Iterbeke et al. (2020) studied, in a secondary school setting, the impact of ability matching and differentiated education in financial education. However, the use of games in financial education has remained an understudied issue. Nevertheless, there is a perception, especially among practitioners, that games can be an important tool in teaching and learning financial literacy (e.g., Maynard et al. 2012). In the broader field of economic education, and especially in research concerning higher education, the potential of using games and simulations has been recognized for quite some time.1 There has been an increased interest in game-based approaches in Finnish schools to increase the effectiveness of financial education. Appropriately for that context, the aim of this study is to evaluate the effects of game-based approaches on financial literacy compared to the situation where there are only traditional teaching methods. Therefore, we do not compare the effects of a stand-alone intervention vs. no intervention but rather, the effects of the delivery mode (game-based vs. traditional) and their influ- ences on financial literacy. The methods that we investigate relate to the paradigm of active or experiential learning, which has been regarded as having promise in economic education (Amagir et al. 2018; Collins and Odders-White 2015). Background Financial literacy teaching in Finnish schools The Finnish educational system has received considerable positive attention related to achievements in the Programme for International Student Assessment (PISA) rankings since the early 2000s. More recently, the performance of Finland in the PISA has somewhat declined. As a response, there has been an increased emphasis on phenomenon-based learning and digitalization in teaching in the new curric- ulum plan for primary and secondary schools (Lonka et al. 2018). Finland also ranked highly in the PISA for financial literacy in 2018, in second place out of 20 countries (Organisation for Economic Cooperation and Development [OECD] 2020). Economic and financial literacy can be regarded as examples of cross-disciplinary themes. Financial literacy is not taught as a separate subject. It is part of a module in “economic knowledge,” which in turn is part of the social studies curriculum. The learning goals of the module include elements both in per- sonal financial management and in a broader macroeconomic context. Economic issues are taught in diverse fields of study, such as study counseling, home economics, and mathematics. Teaching typically takes place within social studies, where teachers mostly specialize in history and not economics. One outcome of this is that teachers actually are not very well informed in economic analysis. Moreover, the material in the school textbooks does not always adequately cover the voluntary K–12 content standards by the Council of Economic Analysis (Siegfried et al. 2010), although they are partly covered in the Finnish curriculum (Kalmi, Maliranta, and Alanko 2019). The ninth year of mandatory schooling in Finland contains an economics curriculum that combines the subjects of financial literacy, which is understood to be personal financial management skills appli- cable at the level of individuals and households, and economic literacy (Hansen, Salemi, and Siegfried 2002; Walstad and Soper 1988), which is related to the application of key economic principles to economic analyses in various issues that are beyond personal financial management; for instance, issues related to informed voting behavior in economic policy contexts. According to the national curriculum for grades 7 through 9, after having the economic knowledge module within social studies, the students should be able to apply economic knowledge in the domains of personal financial management, entrepreneurship and informed career choice, planning for the future, and democratic participation. The curriculum also includes certain attitudinal elements, such as applying ethical thinking in economic choice. However, these normative and attitudinal issues are not graded. EFFECTS OF GAME-BASED FINANCIAL EDUCATION 111 Programs of game-based learning We focused on three distinct game-based programs. The programs have rather different foci, as two of them are geared toward personal financial management and one is a business game. We summarize the characteristics of these games in table 1. Yrityskylä is a business game in a physical learning environment. It is operated through the Economic Information Office (EIO), which is owned by the Confederation of Finnish Industries. Since 2010, EIO has also run a similarly named educational program, Yrityskylä, for primary school students, which is well known in Finland (Kalmi 2018b). The program for the secondary school is more recent, with the pilot phase having started in 2016. Nevertheless, it has spread quite rapidly, and the aim is to increase its scope to a national level in a few years. In the game, students represent a business organization and have several choices. For instance, they can make purchase decisions, try to optimize production processes, apply for loans, make investment decisions, and sell their products, all in a simulated environment. The skills that are required in the game include knowledge about some basic accounting decisions, mathematics, human interaction, and foreign language skills (as some sales pitches are made in English).2 Students play this game in a designated physical environment using digital tools (tablets). They work in teams of five students, which allows for some functional specialization according to student capabilities. A visit to the physical learning environ- ment is preceded by some general lessons on business management and on other decision skills that are required for gameplay. While the topic of the game is not personal finance, the learning goals are related, e.g., to work-life skills and budgeting, so it is quite possible that there is spillover to personal finance domains. Altogether, according to teachers, this game took 5 to 8 hours, including the visit to the physical learning environment that took several hours. Therefore, timewise, this is the most extensive of all the studied programs. Oma Onni is a Web-based learning environment in which the material is produced by students in a vocational school (see Kalmi 2018a). It is sponsored by foundations that own Oma Säästöpankki, a bank with roots in local savings banks. It is produced by the SEDU vocational school in Seinäjoki, Western Finland. It has been in operation since 2010, and its learning contents have been continuously updated. The material in Oma Onni includes games, quizzes, videos, and other interactive material. Most of the learning for secondary school students takes place over the Internet. The students typically go through this material individually in the classroom. Additionally, students from the vocational institute may also visit lower-secondary schools. In addition to being Web-based, another feature of the program is that it represents a form of peer learning, as the lower-secondary school students learn from vocational school students, who are only a few years older. The focus of the program is clearly on personal financial man- agement. The learning goals of the program relate to work-life skills, knowledge that is relevant in managing personal finance, and financial attitudes oriented toward prudence and forward planning. Teachers reported that the use of the learning environment took approximately 4 to 6 hours.3 Money Flow Challenge is a mobile game with five different levels. It is designed by a small personal finance consulting firm called Planago. The game includes topics such as consumption, savings, invest- ment, and mortgage decisions. Students learn to make rational, utility-based decisions concerning per- sonal financial management. This particular learning environment does not require much infrastructure compared to the other two environments and mainly requires a mobile device that can be used for playing Table 1. characteristics of the different game-based programs. Program Yrityskylä oma onni money flow challenge Type of game role play Gamified online learning environment Video game learning environment Physical space online/classroom mobile app mode of working in teams individually, in classroom individually content Business game Personal finance Personal finance learning goals Teamwork skills, communication skills, decision-making calculations Work-life skills, knowledge and attitudes on personal finance Personal finance skills Typical hours used 5–8 4–6 2 Game designer eio/TaT (nonprofit) Sedu (educational organization) Planago (company) 112 P. KALMI AND J. RAHKO the game. Students played this during lectures. There was also a short tutorial. Ideally, it would also include a feedback discussion, although that apparently did not take place in all schools. The hours used for the study of this game were lower than in the two other cases. Students were usually exposed to the Money Flow Challenge over two hours of classes. Money Flow Challenge is also the most recent of the programs, having started in 2014. Game-based teaching is regularly used in many disciplines in Finnish schools. Within economics, one limiting factor has surely been the lack of easy access to relevant games. There are some exceptions, like the monetary policy games developed by the European Central Bank and some games launched by the Finnish branch of Junior Achievement. These games differ from the above-mentioned programs because they are relatively short and easy to use, usually taking up only a fraction of the class hour. According to our teacher survey, the use of other games was somewhat more common in the control group than in the treatment group. In our empirical approach, we test whether this difference affects our results. Study design and methods Allocation of programs In our study, we focus on the development of the economic capabilities of 9th-grade students, who have a mandatory module of economics. The focus of the study is on whether game-based approaches result in better learning achievement than in the absence of gamified teaching. The classes are divided into an intervention group, where the classes participate in game-based teaching, and a control group that does not participate in the interventions. In this approach, it is notable that there exists considerable hetero- geneity due to the different teaching styles of social science educators. Some teachers may well use games in their teaching even if part of the control group. Within the intervention group, the vigor at which teachers incorporate game-based approaches in their teaching is likely to differ among teachers. It is also possible that many teachers in the control group are likely to use active learning approaches, whereas the teaching of many teachers in the intervention group may be more traditional. To control for this, we asked the teachers about the teaching methods they use. The ways in which the programs were allocated to different schools differed somewhat among pro- grams. Yrityskylä and Oma Onni were allocated with direct agreements between the program providers and municipal school authorities. Yrityskylä aims to be a nationwide provider of financial education and is widely used in different municipalities, even though the program for secondary schools is relatively new and not offered in all parts of the country. Oma Onni is in turn tied to the operating area of the savings bank behind the program. In neither of these cases could the researchers affect the allocation of the programs, as the allocation depends on the geographical location of municipalities. In contrast, it was possible to randomize the use of the Money Flow Challenge. A constraint was that we did not want to offer it to schools that already had two interventions to prevent an overload of economic education programs in our groups. Thus, we randomly offered the Money Flow Challenge to 14 schools in the group of schools that either did not participate in the other two interventions or participated in only one of the other interventions. In the end, approximately one-half of the subjects to whom it was offered took it. In summary, because of the practical constraints posed by the arrangements of the programs, even the intention to treat is not randomly allocated (except for the Money Flow Challenge intervention). The intention to treat is determined by external factors such as the availability of the programs that may or may not be related to student characteristics. However, because it is determined at the municipality level and not, e.g., by individual teachers, we have no strong reasons to believe the intervention and control samples differ in their unobservable characteristics. We selected 12 towns for our work, with the idea that we would find a reasonable representation of schools with and without programs. Initially, approximately 60 schools from 12 different towns agreed to participate in the study, but some of them dropped out before the start of the year or during the year. Furthermore, because parental consent letters were required, the number of students was limited to 982 EFFECTS OF GAME-BASED FINANCIAL EDUCATION 113 of over 3700 students who took the pre-intervention survey. Due to school, class, or student attrition, the final sample size was further limited to 640 students who also took the post-intervention survey. Thus, in the final data, we analyzed pupils from 42 schools in 11 towns. A total of 433 of the responding students received game-based teaching along with typical economics education; an additional 44 students should have received the treatment but did not, while 163 did not receive treatment and formed our control group.4 Some students participated in two game-based interventions, while others participated in only one intervention. Table 2 presents the final data for the number of students in different groups. Survey data and questionnaire We conducted pre- and post-intervention surveys on economic knowledge, savings behavior, planning ahead, and impulse shopping among ninth year students. The survey was based partly on a previous questionnaire used in the Oma Onni program in analyzing student learning (Kalmi 2018a). However, the Oma Onni program focuses solely on personal finance, whereas some programs included in our study, as well as the learning goals of the Finnish curriculum, are broader, including issues of “economic literacy” (for instance, firms, macroeconomics, and economic policy). Therefore, we broadened these questions so that one-half of the knowledge questions covered questions related to personal finance, and the other half of the questions concerned firms and macroeconomics.5 Moreover, the questionnaire was divided into knowledge questions6 and into questions dealing with behavior and attitudes. The question sets in the two surveys were identical regarding financial behavior, but the financial knowledge questions were different. The two surveys and their questions were pretested with vocational education students. The first survey was conducted in autumn 2017 with the assumption that no economics education took place before the survey. The second survey took place in spring 2018 after the educational inter- ventions and other economics teaching in the curriculum. However, some classes had already started economics education before they took the first survey, while some classes had their final lessons in economics only after the second survey was taken. Therefore, we obtained the teaching schedules and controlled for these differences in our empirical setting. We also conducted a survey among the parents in which we asked for background information, such as parental education, and tested for parents’ own financial knowledge with the “three big” questions of financial literacy (Lusardi and Mitchell 2014). The additional parental survey was voluntary. We had a response rate of 76 percent of those who also gave a research permit. Our pre- and post-intervention surveys contained questions on financial knowledge and attitudes and behavior, as well as students’ background characteristics. Table 3 presents definitions and measure- ments of the variables used in the empirical analysis.7 Our main outcome variable is the standardized test score, which measures the student’s financial knowledge. The test scores were demeaned so that they had a mean of zero and a standard error of one. Because some of the schools and classes dropped out between the first and second survey and parental consent was required, our final sample was shaped by a two-stage selection process and, thus, may not present the initial target group of the study. Furthermore, some parents selected not to respond to the parental survey. Table 4 analyzes these attrition processes by showing students’ initial level of financial knowledge according to the group. Table 2. Sample size by group. Group number of Students control group 163 Treatment group 477 1 intervention 208 2 interventions 269 Yrityskylä 359 money flow challenge (mfc) 176 oma onni 211 Total 640 114 P. KALMI AND J. RAHKO Table 4. attrition process and students’ financial knowledge in the pre-survey. Treatment Group control Group mean Sd observations mean Sd observations all students* −0.184 1.030 2,603 −0.267 1.088 1,137 Both surveys returned −0.105 0.974 1,572 −0.096 1.102 498 Both surveys and parental consent returned −0.018 0.993 477 0.053 1.020 163 Both surveys, parental consent and parental survey returned 0.024 0.997 371 0.105 1.016 117 *difference between the treatment and control group is statistically significant at the 5% level. Table 4 shows that students in the treatment group performed slightly better in the pre-intervention survey. The table further illustrates that those students who also returned the second survey performed significantly better in the pre-intervention survey, which may reflect unobservable school or teacher characteristics. This selection process appears in both the treatment and the control groups. However, a smaller proportion of students in the control group answered the post-intervention survey than in the treatment group, and the attrited students in the control group appear to differ from non-attrited students somewhat more than in the treated group. We find no statistically significant differences in the financial knowledge between the treatment and control groups for students who took both surveys. Students for whom we obtained parental consent also performed significantly better in the pre-intervention survey. Receiving a response to the parental survey also appears to correlate with students’ higher financial knowledge. The plausible reason for this is that answering the parental survey correlates with the student’s socioeconomic background. Parents who take a more active interest in their children’s school performance can be both more likely to respond to parental surveys and support their children to attain higher levels of academic performance. However, we do not believe that these selection processes endanger our treat- ment effect estimation because these processes appear in both groups. As attrition is more common in the control group, we explore the predictors of attrition and use inverse probability weighting in our regressions to take selected attrition into account. Finally, it should be noted that our final sample includes students with a higher initial level of financial knowledge than the whole population of 9th-graders, which may limit how far we can generalize our results.8 Table 3. definition of variables. Variable description Standardized test score demeaned number of correct answers in the financial knowledge test Standardized test score, personal finance demeaned number of correct answers in the personal finance questions Standardized test score, macroeconomic knowledge demeaned number of correct answers in the macroeconomic knowledge questions Swedish 1 if student answered the survey in Swedish, 0 otherwise math grade Student’s previous math grade average grade Student’s grade average in the previous school report Talks about money 1 if student talks about financial matters with parents, 0 otherwise Bank account 1 if student has a bank account, 0 otherwise Pocket money 1 if student receives pocket money from parents, 0 otherwise over 100 books 1 if there are over 100 books at home, 0 otherwise Parent’s financial literacy number of correct answers in the parental financial literacy test (0–3) Teaching started 1 if the teaching of economics had started before class took the first survey, 0 otherwise Teaching continues 1 if the teaching of economics had not finished before class took the post-survey, 0 otherwise income 1 if student has earned income, 0 otherwise Knows how much money on account 1 if student knows how much money he/she has in their bank account, 0 otherwise Saves regularly 1 if student saves regularly for some purpose, 0 otherwise Savings 1 if student saves money in a bank account, at home or otherwise, 0 otherwise Saves extra money 1 if student saves money that he/she receives from parents, 0 otherwise finds saving easy 1 if student finds saving easy or quite easy, 0 otherwise finds saving profitable 1 if student finds saving money profitable, 0 otherwise Plans ahead 1 if student finds that planning ahead is the best way to achieve goals, 0 otherwise finds personal finance easy Student finds personal finance easy, values 1–5 Plans the use of money Student plans the use of money ahead of time, values 1–5 impulse shopping Student buys impulsively, values 1–5 interest in economic issues Student finds economic issues interesting, values 1–5 Saving questions combined Sum of the five savings-related questions, values 0–5 EFFECTS OF GAME-BASED FINANCIAL EDUCATION 115 Table 5 presents summary statistics for the treatment and control groups based on the first survey before any intervention had taken place. The initial level of financial knowledge does not statistically differ significantly between the two groups. The groups are also similar with respect to variables reflecting family background. However, the groups differ with respect to some observable background character- istics, e.g., compared with the treatment group, the control group had more Swedish-speaking students, lower average math grades, and the teaching of economics had already started in more classes. There are also some initial differences in students’ attitudes toward saving and personal finance. These are observ- able characteristics that we can control in our empirical setting. Estimation methods Next, we specify a regression model to investigate the effect of game-based economics education on financial knowledge and behavior. As we explained before, the programs were not randomly placed in municipalities, so a difference-in-differences estimation strategy is appropriate. We compare the change in outcome between the first and second surveys across treatment and control groups while controlling for students’ and classes’ background characteristics. The basic specification is the following: y Intervention Post Intervention Post Contri t i t it, *        + + olsit it , (1) where y refers to the outcome variable, which is either the level of financial knowledge or selected financial behavior outcome. The Intervention variable indicates whether student i is in the treatment or control group. In some of the specifications, we divide the treatment group further, depending on whether the student participated in one or two interventions. We also estimate the disaggregate results for each of the three interventions. The Post variable is a time variable for the separation between the pre-survey and post-survey. Our main interest is in the interaction of the Intervention and Post variables, which reveals the treatment effect of the game-based education interventions. Control variables include Table 5. Baseline (pre-survey) summary statistics. Treatment Group* control Group** difference in means Test p-valuemean Sd mean Sd Standardized test score −0.018 0.993 0.053 1.020 0.437 Standardized test score, personal finance 0.009 1.013 −0.028 0.964 0.677 Standardized test score, macroeconomic knowledge −0.036 0.989 0.104 1.027 0.131 male 0.501 0.501 0.466 0.500 0.444 Swedish 0.031 0.175 0.190 0.394 0.000 math grade 8.195 1.353 7.890 1.410 0.017 average grade 8.339 0.904 8.183 0.915 0.061 Talks about money 0.581 0.494 0.521 0.501 0.192 Bank account 0.784 0.412 0.798 0.403 0.714 Pocket money 0.463 0.499 0.448 0.499 0.733 over 100 books 0.539 0.499 0.528 0.501 0.806 Teaching started† 0.106 0.309 0.218 0.414 0.007 Teaching continues† 0.152 0.359 0.202 0.403 0.220 Parent’s financial literacy‡ 2.477 0.806 2.479 0.826 0.986 income 0.604 0.490 0.681 0.468 0.073 Knows how much money on account 0.841 0.366 0.840 0.367 0.996 Saves regularly 0.545 0.498 0.626 0.485 0.070 Savings 0.945 0.227 0.963 0.189 0.329 Saves extra money 0.537 0.499 0.552 0.499 0.733 finds saving easy 0.725 0.447 0.779 0.416 0.163 finds saving profitable 0.958 0.201 0.963 0.189 0.769 Planning ahead 0.843 0.364 0.840 0.367 0.946 finds personal finance easy 3.855 0.909 4.006 0.790 0.044 Plans the use of money 3.296 1.152 3.399 1.152 0.325 impulse shopping 2.321 0.981 2.221 0.969 0.258 interest in economic issues 3.547 1.176 3.736 1.127 0.069 Saving questions combined 3.711 1.204 3.883 1.135 0.100 Notes: *477 observations; **163 observations; †376 observations in the treatment group and 124 observations in the control group; ‡371 observations in the treatment group and 117 observations in the control group. 116 P. KALMI AND J. RAHKO background variables related to student, class, and family characteristics. Prior studies have observed that students’ numeracy and cognitive skills, as well as socioeconomic background, are related to financial knowledge and behavior (e.g., Frisancho 2020; Lührmann, Serra-Garcia, and Winter 2015); thus, we control for these characteristics. These include gender, mother tongue, math grade, and average grade, having a bank account, and receiving pocket money. We also include as family background characteristics the number of books at home, whether the student talks about money with parents, and parental financial literacy. The control variables are measured in the first survey to avoid any influence from the interven- tions on the control variables. The difference-in-differences estimation rests on the assumption that, absent intervention for both treatment and control groups, students will have similar development in their financial knowledge. All students follow the same curriculum in Finnish schools and, thus, the parallel trend assumption should hold. However, some of the classes had already started their economics education prior to the first survey, and some classes finished their economics lessons only after the second survey. These differences may cause the parallel trends assumption to not hold. However, for most classes, we have this information available, and we can thus add a control variable as well as a time interaction variable to allow for different time trends with those particular students. Therefore, observations where this information is not available are dropped in some of the regressions. We identify the classes and students who, according to a preassignment, should have received the treatment. However, some classes did not follow through with the game-based interventions. For example, the Money Flow Challenge game was randomly offered to schools, but some teachers self-selected not to use it in their teaching. Participation in Yrityskylä was preassigned at the municipality level; however, some schools or classes did not participate, e.g., due to scheduling reasons. Thus, our empirical analysis estimates the average effect of intention to treat (ITT). We can also identify students who actually par- ticipated and received the treatment; i.e., we could estimate the average treatment effect on treated (ATT). If treatment participation reflects teachers’ self-selection, participation can be correlated with the knowl- edge and behavioral outcomes that we study. Therefore, ATT results can be biased, but ITT results provide an unbiased picture. However, as Ding and Lehrer (2010) discuss, selective attrition of students may bias even ITT estimates. Following their approach, we thus use inverse probability-weighted regressions to correct for potentially selective attrition. First, let’s define Lt+1 = 1 to indicate students who did not answer the post-intervention survey and Lt+1 = 0 to indicate non-attrited students. Then, we may model the attrition process as follows: L L Controls ut i t i i i   1 11, , * if >0 and zero otherwise (2) where Lt i+1, * is a latent variable, Controls indicate variables from the pre-intervention survey that predict student attrition, and u is an error term. We estimate this equation with a probit model and then calculate the probability of each student’s non-attrition, i.e., Pr(Lt+1,I = 0) given the pre-intervention survey information. We then use the inverse of this predicted probability to weight the observations used to estimate equation 1. In our data, the attrition may happen at the school, class, or student level. Of 982 students who answered the pre-intervention survey altogether, 342 did not answer the second survey. Of these students, 200 did not answer the second survey either because their school or class dropped out. One hundred forty-two students did not answer, even though some of their classmates did. Thus, it appears that school, class, or teacher characteristics are more important determinants of attrition than individual student characteristics. This was confirmed when we analyzed which factors explained the attrition. Treatment group status, the language of the school, and whether teaching had started were the most significant drivers of student attrition. Lower pre-survey financial knowledge predicted the attrition in some regressions, but its effect varied depending on whether other controls were included. Other characteristics of the students did not explain the probability of attrition. Thus, we chose to use treatment status, language, and pre-intervention survey financial knowledge to calculate the probability of attrition. As teaching schedules and parental financial knowledge are used as controls in some regressions, we also use these variables to calculate the inverse probabilities for those ITT estimations, where these variables are used. Furthermore, when we EFFECTS OF GAME-BASED FINANCIAL EDUCATION 117 separate between the intervention programs or one vs. two interventions in equation 1, we use these same treatment indicators to explain attrition. The results for the probit models explaining the attrition are available upon request. As a robustness test to account for student dissimilarity between the intervention and control group, we also use propensity score matching and difference-in-differences estimation. However, because the theory developed by Abadie and Imbens (2006, 2016) has not been extended to handle multivalued treatments, we cannot use matching techniques in our main analysis to compare one vs. two interventions or the effects of the three different programs. Therefore, we only use matching as a robustness test for the case of any treatment vs. no treatment. Results Financial knowledge Table 6 presents the estimation results from difference-in-differences estimations using inverse probability weighting to explain students’ financial knowledge.9 First, we analyze students’ overall performance on the financial knowledge questions and add more control variables in the second specification. In spec- ifications 3 and 4, we separate whether students participated in one or two interventions. Then, we separately analyze how their knowledge developed in the two subfields of our survey: personal finance (specifications 5 and 6) and firms and macroeconomics (specifications 7 and 8). In table 6, we present only the estimates for the main variables of interest. All the control variables are measured at the time of the first survey, and thus, control variables related to financial behavior have not been influenced by the potential treatment.10 From table 6, we can observe that after controlling for student background, the treatment group performs somewhat worse than the control group in the pre-survey. However, the treatment effect esti- mate is positive and significant, suggesting that the treatment group caught up and performed better than the control group in the second survey. The average treatment effect suggests an improvement in financial knowledge by 0.342 standard deviation in the intervention group. The size of improvement is in line with the effects reported by Kaiser and Menkhoff (2020) for financial literacy interventions in general. Table 6. Pooled financial knowledge results. average treatment effect of the intention to treat. financial Knowledge Personal finance macroeconomics 1 2 3 4 5 6 7 8 Post 0.064 0.083 0.066 0.070 −0.212** −0.226** 0.287** 0.275** (0.080) (0.107) (0.079) (0.105) (0.098) (0.094) (0.117) (0.116) Treatment −0.152 −0.213 −0.162 −0.217* (0.154) (0.130) (0.124) (0.126) Post*Treatment 0.342*** 0.415*** 0.419*** 0.348** (0.102) (0.129) (0.121) (0.135) Treatment(1) −0.073 −0.175 −0.143 −0.171 (0.165) (0.122) (0.123) (0.116) Post*Treatment(1) 0.251** 0.336** 0.331** 0.287* (0.114) (0.152) (0.156) (0.143) Treatments(2) −0.209 −0.244 −0.175 −0.256 (0.160) (0.166) (0.151) (0.166) Post*Treatments(2) 0.414*** 0.482*** 0.492*** 0.400** (0.114) (0.139) (0.125) (0.154) Student characteristics x x x x x x x x Timing of teaching x x x x x x Parent’s financial literacy x x x x x x N 1280 752 1280 752 752 752 752 752 adjusted r-squared 0.198 0.233 0.197 0.231 0.180 0.178 0.230 0.228 equality of iTT estimates, p-value 0.165 0.308 0.297 0.410 Notes: inverse probability-weighted regressions. Standard errors are clustered by school and are presented in parentheses. *p < 0.10; **p < 0.05; ***p < 0.01 118 P. KALMI AND J. RAHKO Table 7. intervention-specific financial knowledge results. financial Knowledge Personal finance macroeconomics 1 2 3 4 5 6 Post 0.074 0.081 −0.159* −0.218** 0.233*** 0.288** (0.074) (0.098) (0.082) (0.089) (0.067) (0.109) Yrityskylä −0.141 −0.204** −0.083 −0.175* −0.161 −0.194** (0.110) (0.095) (0.094) (0.096) (0.114) (0.095) Post*Yrityskylä 0.307*** 0.409*** 0.308*** 0.473*** 0.259** 0.298** (0.098) (0.118) (0.095) (0.109) (0.101) (0.125) mfc −0.236** −0.290*** −0.058 −0.142 −0.330*** −0.354*** (0.093) (0.089) (0.079) (0.088) (0.105) (0.097) Post*mfc 0.060 0.156 0.015 0.110 0.084 0.166 (0.114) (0.131) (0.105) (0.111) (0.121) (0.147) oma onni −0.010 0.091 0.039 0.058 −0.044 0.101 (0.091) (0.128) (0.083) (0.126) (0.093) (0.121) Post*oma onni 0.165* 0.091 0.162 0.055 0.141* 0.103 (0.082) (0.095) (0.098) (0.094) (0.078) (0.108) Student characteristics x x x x x x Timing of teaching x x x Parent’s financial literacy x x x N 1280 752 1280 752 1280 752 adjusted r-squared 0.199 0.240 0.125 0.179 0.214 0.241 equality of iTT estimates, p-value 0.168 0.116 0.104 0.030 0.400 0.418 Notes: inverse probability-weighted regressions. Standard errors are clustered by school and are presented in parenthesis. *p < 0.10; **p < 0.05; ***p < 0.01 In specification 2, we also consider that the teaching started and finished at different times in different classes and also control for parent’s financial literacy. Classes where teaching had already started per- formed significantly better in the first survey and subsequently improved their scores less in the second survey (results available from the authors). Including these controls increases the main coefficient of interest (Intervention * Post) slightly. Based on previous literature, we also expected that parental education would be an important control variable explaining student performance. However, in our results, this is not the case; thus, in the end, we did not include parental education among the control variables. We suspect that the lack of explanatory power may be because obtaining parental consent and answers in the parental survey already correlate with the family’s socioeconomic background; thus, parental education does not contain additional explanatory power. In specifications 3 and 4, we separately studied the treatment effect of one game-based intervention and that of two game-based interventions. The estimate of one intervention is positive, and the estimate of two interventions is higher and more strongly significant. We also tested whether these effects are of equal size. The F-test p-values are reported at the bottom of table 6. However, the null hypothesis of equal ITT effects cannot be rejected. In specifications 5 through 8, we studied whether the interventions impact personal finance and macroeco- nomic and firm-related knowledge differently. Overall, we observe that the interventions significantly improve students’ knowledge in both subfields and that the treatment effect estimates are similar in both cases. In table 7, we analyzed the three interventions separately. In our specifications, we did not allow for complementarity or substitutability between the interventions because, otherwise, the number of inter- action terms would grow excessively. From table 7, it is apparent that the initial level of financial knowl- edge varies by the intervention group, most notably for the group that was assigned to use Money Flow Challenge (MFC). This is surprising, considering that this is the only group where the treatment assign- ment was randomized. Turning to treatment effect estimates, we notice that the treatment effect of the Yrityskylä game is positive and strongly significant. The estimates are similar to those in table 6. The treatment effect of Oma Onni is positive and significant at the 10 percent level in some specifications, but not when we include a full list of control variables in the estimation. Finally, the treatment effect of the Money Flow Challenge is not significant in any of our model specifications, although the point estimate is often of similar size to Oma Onni. Moreover, the F-test p-values at the bottom of table 7 show that the null hypothesis of equality of these three ITT estimates can be rejected only for personal finance-related questions. EFFECTS OF GAME-BASED FINANCIAL EDUCATION 119 The differences in the knowledge gains between the interventions are consistent with the amount of time used in the interventions (see also table 1). For instance, the students used the most time for Yrityskylä and the least time for Money Flow Challenge. Yrityskylä also has the biggest resources by far for pedagogical development and implementation of all these programs, which also might be reflected in its results. However, as Yrityskylä focused more on business life and work-life skills, it is somewhat surprising that its effects are even stronger on personal finance-related knowledge. This can perhaps be interpreted as transfer learning: for a successful program, the learning effects can spill over from one domain to another related domain (Kneppers et al. 2007). For the other two programs, we do not observe clear differences between personal finance and macroeconomic questions. Financial behavior and attitudes Students’ level of financial knowledge correlates significantly with most aspects of self-reported financial behavior (correlations available from the authors). However, when we analyze whether the education intervention altered students’ financial behavior, we observe that the treatment effects are negligible and statistically insignificant, with only a few exceptions. Table 8 presents the estimation results from differ- ence-in-differences estimations regarding the behavioral variables. The outcome variables are binary variables; thus, we use probit estimation and report average marginal effects in table 8. In these estima- tions, we use the following control variables: gender, language, math grade, average grade, and whether there are over 100 books at the student’s home.11 Based on the results in table 8, we can observe that only in two of the savings questions (Savings and Finds saving profitable) is the treatment effect significant. When we separate between one and two inter- ventions, we somewhat puzzlingly obtain a negative treatment effect for two treatments in the Saves regularly as well as Bank account questions but a positive effect in the Savings and Finds saving profitable questions. In conclusion, we find no clear effects on savings behavior. In table 9, we report the further treatment effect estimates for financial behavior and attitudes. These variables are ordinal variables. The first four variables (Finds personal finance easy, Plans the use of money, Impulse shopping, Interest in economic issues) take values between “1” and “5,” where “1” means that the student strongly disagrees with the statement and “5” means that the student strongly agrees with the statement. The final variable is formed by aggregating all five savings-related variables in table 8. Thus, this variable takes values between 0 and 5, where high values indicate that the student has answered positively to more savings questions. The results are based on an ordered probit estimation. To save space, Table 8. financial behavior results. average treatment effect of intention to treat. income Bank account Knows how much money on account Pocket money Saves regularly Post 0.032 0.032 0.072*** 0.072*** 0.071* 0.071* 0.020 0.020 0.075 0.075 (0.028) (0.028) (0.026) (0.026) (0.037) (0.037) (0.028) (0.028) (0.058) (0.058) Treatment −0.087 −0.001 0.023 0.055 −0.039 (0.059) (0.027) (0.028) (0.043) (0.060) Post*Treatment 0.005 −0.015 −0.034 −0.030 −0.101 (0.040) (0.037) (0.041) (0.033) (0.062) Treatment(1) −0.154** −0.054* −0.012 0.044 −0.054 (0.062) (0.030) (0.034) (0.058) (0.074) Post*Treatment(1) 0.055 0.058 −0.027 −0.019 −0.060 (0.058) (0.036) (0.041) (0.036) (0.065) Treatments(2) −0.021 0.053** 0.060* 0.065 −0.028 (0.060) (0.027) (0.031) (0.051) (0.057) Post*Treatments(2) −0.039 −0.080** −0.041 −0.038 −0.135** (0.037) (0.038) (0.049) (0.036) (0.064) Student characteristics x x x x x x x x x x Pseudo r-squared 0.020 0.025 0.019 0.027 0.014 0.021 0.016 0.016 0.018 0.018 equality of iTT estimates, p-value 0.088 0.000 0.722 0.536 0.061 120 P. KALMI AND J. RAHKO Table 8. (continued). Savings Saves extra money finds Saving easy finds Saving Profitable Planning ahead Talks about money Post −0.055*** −0.055*** 0.054* 0.054* −0.010 −0.010 −0.023 −0.023 −0.024 −0.024 0.055 0.055 (0.016) (0.016) (0.029) (0.029) (0.023) (0.023) (0.015) (0.015) (0.039) (0.039) (0.048) (0.048) Treatment −0.026 −0.052 −0.093** −0.005 −0.014 0.045 (0.017) (0.046) (0.041) (0.013) (0.031) (0.048) Post* Treatment 0.058*** −0.030 0.043 0.034* 0.018 0.050 (0.019) (0.042) (0.029) (0.019) (0.044) (0.053) Treatment(1) −0.029 −0.055 −0.093** 0.002 −0.007 0.047 (0.018) (0.057) (0.041) (0.019) (0.034) (0.046) Post* Treatment(1) 0.057** −0.007 0.033 0.019 −0.007 0.062 (0.027) (0.055) (0.033) (0.018) (0.048) (0.058) Treatments(2) −0.023 −0.051 −0.092** −0.010 −0.018 0.042 (0.019) (0.047) (0.047) (0.013) (0.036) (0.061) Post* Treatments (2) 0.058*** −0.049 0.052 0.045* 0.041 0.040 (0.018) (0.049) (0.035) (0.025) (0.049) (0.055) Student charac teristics x x x x x x x x x x x x Pseudo r-squared 0.039 0.039 0.041 0.042 0.047 0.047 0.057 0.060 0.043 0.044 0.017 0.017 equality of iTT estimates, p-value 0.981 0.491 0.589 0.260 0.243 0.600 Notes: inverse probability-weighted regressions. marginal effects after probit estimation. 1280 observations. Standard errors are clustered by school and are presented in parenthesis. *p < 0.10; **p < 0.05; ***p < 0.01 we report only the marginal effects for the treatment variables. The marginal effects describe the increase in the likelihood that students belong to a particular group. According to table 9, we can observe that game-based financial education interventions positively affect students’ self-reported interest in economic issues. This finding appears to be related to the fact that these students start from a lower level of interest, but they catch up by the time of the second survey. Students who participated in one treatment also report that they plan their use of money in the post-sur- vey more often than those in the control group. With respect to the other behavioral variables, we do not observe statistically significant treatment effects. We also separately estimated the financial behavior for the three interventions. In line with tables 8 and 9, these estimations did not reveal systematic results of the effectiveness of the individual programs. The results are available from the authors. Overall, we conclude that even though game-based education interventions improve students’ financial knowledge, we find only a very weak impact on their financial behavior and attitudes. False discovery rates As previously described, we have analyzed the effects of several different interventions on various knowl- edge and behavioral outcomes. This raises concerns of multiple inferences and that some statistically significant results may occur by chance and, thus, do not reflect real treatment effects. To consider this possibility, we calculated the False Discovery Rate (FDR) q-values following Anderson (2008). The FDR is the proportion of rejected null hypotheses that are false rejections (type I errors). Thus, the q-values can be interpreted analogously to the standard p-values. For the statistically significant treatment effect estimates in tables 6 and 7, the corresponding q-values were less than 0.1 except for the treatment effect of a single program in columns 6 and 8 in table 6 and the treatment effects of the Oma Onni program in columns 1 and 5 in table 7. Thus, our main findings that the game-based interventions have a positive effect on financial knowledge and that two interventions appear to work better than one remain valid after considering multiple inferences. EFFECTS OF GAME-BASED FINANCIAL EDUCATION 121 With respect to the behavioral outcomes in table 8, the q-values of the statistically significant treatment effects were less than 0.1 for other outcome variables, but not for Finds saving profitable. We also observed unexpected negative treatment effects when we separated between one and two interventions in table 8. These negative treatment effects had q-values of 0.099, which could support our conclusion that these negative effects are likely to be false discoveries. Finally, in table 9, the q-values of the statistically signif- icant treatment effects were less than 0.1, except for Impulse shopping and Interest in economic issues for participation in a single intervention. Table 9. financial behavior and attitudes results. average treatment effect on treated students. ordered probit estimation. finds Personal finance easy Plans the use of money impulse Shopping interest in economic issues Saving Questions combined Treatment 0 0.006** 1 0.003 0.009 −0.026 0.025* 0.011* 2 0.032** 0.050 −0.008 0.051** 0.030** 3 0.017* 0.001 0.003 0.002* 0.026** 4 0.006 −0.028 0.024 −0.014* −0.001 5 −0.057** −0.032 0.006 −0.064** −0.072** Post*Treatment 0 0.000 1 −0.002 −0.007 −0.034 −0.028** 0.001 2 −0.017 −0.040 −0.011 −0.057*** 0.002 3 −0.009 −0.001 0.004 −0.003*** 0.002 4 −0.003 0.022 0.032 0.016** −0.000 5 0.031 0.025 0.008 0.071** −0.005 Treatment (1) 0 0.006** 1 0.003 0.016** −0.021 0.019 0.012* 2 0.028 0.091** −0.007 0.038 0.032* 3 0.014 0.001 0.003 0.002 0.027* 4 0.005 −0.050** 0.020 −0.011 −0.001 5 −0.050 −0.058** 0.005 −0.048 −0.075* Post*Treatment(1) 0 −0.001 1 −0.001 −0.016** −0.046 −0.020 −0.001 2 −0.011 −0.090** −0.015 −0.041* −0.004 3 −0.006 −0.001* 0.006 −0.002* −0.003 4 −0.002 0.050** 0.043* 0.011* 0.000 5 0.020 0.058** 0.011 0.051* 0.009 Treatments(2) 0 0.006** 1 0.003 0.003 −0.030 0.031* 0.011* 2 0.035** 0.015 −0.009 0.062** 0.029** 3 0.018** 0.000 0.004 0.003* 0.025** 4 0.006 −0.008 0.028 −0.018* −0.001 5 −0.063** −0.009 0.007 −0.079** −0.070** Post*Treatments(2) 0 0.001 1 −0.002 0.000 −0.023 −0.035** 0.003 2 −0.022 0.002 −0.007 −0.069*** 0.007 3 −0.012 0.000 0.003 −0.003** 0.006 4 −0.004 −0.001 0.022 0.020** −0.000 5 0.040 −0.001 0.006 0.088*** −0.017 Pseudo r-squared 0.035 0.035 0.016 0.018 0.025 0.025 0.028 0.029 0.018 0.018 equality of iTT estimates, p-value 0.581 0.008 0.384 0.184 0.407 Notes: inverse probability-weighted regressions. marginal effects after ordered probit estimation. 1280 observations. *p < 0.10; **p < 0.05; ***p < 0.01 Table 10. PS matching results. financial Knowledge Personal finance macroeconomics aTT 0.314** 0.306** 0.270** (0.123) (0.136) (0.125) Notes: Kernel-based propensity score matching and difference-in-differences estimation with a common support. *p < 0.10; **p < 0.05; ***p < 0.01 122 P. KALMI AND J. RAHKO Robustness tests Table 4 lists significant imbalances in the pretreatment covariates between the treatment and control groups, which can compromise the parallel trends assumption. We controlled for these observable dif- ferences in our baseline estimations, and we now conduct further robustness checks. First, we estimated the treatment effect by applying propensity score (PS) matching and differ- ence-in-differences estimation. The first step was to calculate the probability of treatment using probit estimation (results available from the authors). We estimated the probability conditioning on gender, language, math grade, average grade, and whether the teaching had started. The choice of variables was based on the observed differences in table 5. All variables are statistically significant in the probit esti- mation, except for math and average grade, which are individually insignificant but jointly significant. We do not include other variables in the probit estimation because including additional but statistically not significant regressors may lead to high variance and problems with the common support condition (Caliendo and Kopeinig 2008). The matching reduces the differences in the covariates, and the remaining differences are statistically insignificant. After the estimation of the PS, we estimated the average treatment effect on the applicable students using difference-in-differences estimation and kernel matching. The results are presented in table 10. The PS matching results show that the treatment effect on financial knowledge is approximately 0.3 standard deviation. The estimate is statistically significant but slightly lower than the estimates in table 5, where the estimates are in the range of 0.35 to 0.42. Further tests, which used coarsened exact matching on the same set of variables, again confirmed the main results. We also studied the effects on behavioral outcomes using the matching methods previously described. (These results are available from the authors.) PS matching showed statistically significant effects on the following outcomes: Savings, Finds personal finance easy, and Interest in economic issues. The first and third results are in line with tables 8 and 9, but the second result is new. The validity of the parallel trends assumption also could be compromised if teaching in the control group and treatment group differed in other respects outside of the game-based interventions. We also asked teachers about the number of hours used for teaching and whether any other games were used to see whether the treatment and control groups differed. The use of other games was more common in the control group. Furthermore, control group teachers used slightly more hours to teach economics. Thus, we included these variables and their interaction with Post in the difference-in-differences esti- mations. However, these variables were not statistically significant and did not influence the main coef- ficient of interest (results available from authors). It is worth noting that the other games that the teachers mention are short games that take only a fraction of study lessons to play. Thus, it is plausible that their effect would be small. Another difference between the students is their mother tongue, and thus, the study material they use in class. As table 5 indicates, Swedish-speaking students form a larger share of the control group than the treatment group. In the PS matching above, we matched the students based on language as well as other characteristics. Furthermore, we re-estimated table 6 without the Swedish-speaking students. There was very little change in the treatment effect estimates. Finally, there is attrition in our sample because all students did not answer the post-survey. We accounted for this, using inverse probability weighting in our regressions. As a further analysis of the implications of this attrition, we conducted unweighted regressions and constructed bounds on the treatment effects on financial knowledge following Drexler, Fischer, and Schoar (2014). While some of the treatment effects in tables 6 and 7 are sensitive to the assumptions about the attrition process, this is not the case when the full list of control variables is included or when we analyze the effects of partici- pation in two interventions or in Yrityskylä program.12 Conclusions In this article, we studied the impact of game-based interventions in teaching economics relative to the impact of traditional teaching methods in Finnish lower-secondary schools. Our findings indicate that EFFECTS OF GAME-BASED FINANCIAL EDUCATION 123 learning outcomes are better with game-based interventions when the relevant measure is economic knowledge. This applies both to personal finance questions as well as to more macro- and firm-oriented questions. There is also some evidence that game-based interventions positively impact the interest of students in economic issues. However, the impacts on (self-reported) behaviors are weak. Methodologically, the challenge is that student behaviors are hard to measure for under-aged students who do not make important economic decisions independently. Future studies could investigate how interventions could influence behavioral outcomes within game-based simulations. The study took place in the context of Finnish 15-year-old students. The academic performance for this group has been found to be relatively high compared with other countries, such as the United States. The recent OECD (2020) study showed this comparison also to be true for financial literacy. The question arises as to what extent the results can be generalized to other settings. However, the external validity problem of the samples drawn from certain contexts is almost always present in empirical work. Although most published studies on economic education have been conducted in the United States, studying a European country may provide further validity to the research on economic education. Additionally, the specific elements in economic games in the Finnish context that would question the generalizability of the results are not clear. The game-based approaches described in this article have their counterparts in other countries as well, including the United States. While similar results have been obtained in prior literature, a novelty of our findings is that instead of focusing on a situation where there is a games intervention vs. an absence of intervention, these find- ings relate to comparisons between two different types of teaching approaches. The findings strongly support pedagogical innovations in the field of personal finance and economics. The use of game-based learning approaches, which are a subset of active learning approaches, shows promise in simultaneously making studying economics more practical and fun. The strongest learning effects appear to be related to a game that students play in teams and that has diverse learning objectives, including interpersonal and communication skills. This game can also be related to content other than just personal finance. An interesting question for future studies of the use of games in financial education is to see whether direct approaches work best or whether learning is enhanced by only tangentially participating in games that relate to financial literacy. The direct approach is taken by many available mobile games, while there are examples of indirect approaches as well, e.g., economics applications in blockbuster games such as Minecraft or The Sims. Funding This work was supported by the research grants 305292 and 327241 from the Academy of Finland. The Academy of Finland did not participate in the research in any other than funding role, and it has no independent interest in the research results. Notes 1. For an early evaluation study, see Gremmen and Potters (1997) and for an early review of the use of games in economics, see Sutcliffe (2002). 2. See description at https://yrityskyla.fi/en/activity/. 3. See https://omaonni.fi/ (in Finnish only). 4. We are not aware that any of the students in the control group had actually received the treatment (i.e., participated in the designated game-based projects). 5. Some of the questions in the macroeconomics section dealt with interest rates and inflation, which also could be included among personal finance questions. 6. The surveys contained 60 questions that tested economic knowledge. 7. While behavioral variables can be criticized on the grounds that students in the studied age group seldom make important independent economic decisions, including such measures even in school studies is common (see Table A1 in Kaiser and Menkhoff 2020). The use of such measures is common even among primary school students (see, e.g., Batty, Collins, and Odders-White 2015). Childhood financial behaviors are often reflected in adult financial behaviors (Brown and Taylor 2016). 124 P. KALMI AND J. RAHKO 8. However, we also tested whether the treatment effects were heterogeneous with respect to students’ academic performance. We did not find any statistically significant heterogeneity and, thus, do not report these results. 9. Unweighted difference-in-differences estimations yielded very similar results. 10. The full-result table with coefficient estimates for the control variables is available on request from the authors. 11. We do not include the following as control variables: whether the student talks about financial matters at home, has a bank account, or receives pocket money from parents, because these reflect student’s behavior and are potential outcome variables. 12. The results are available upon request. Acknowledgments The authors are thankful for the generous funding from the Academy of Finland (research grants 305292 and 327241); to the editors and two anonymous referees of this journal and to Tim Kaiser for useful comments; to partners (Yrityskylä, Oma Onni, Money Flow Challenge) and especially to Kaisa Koistinen, Tiina-Maija Toivola, Tomi Alakoski, Ari Autio, Mikko Tavi, and Marko Vänskä; to research collaborators Tanja Sihvonen, Olli Raatikainen, Saija Alanko, Sanna Peltonen, Janne Pekkala, Jussi Loukiainen, Eusebio Scornavacca, Andy Lymer, Peter Davies, and Ari Hyytinen; and to the participants at the following conferences for their help, discussions and other forms of support: Association of the European Economics Education in Copenhagen, August 2018, Council for Economic Education, Los Angeles CA, October 2019, NAEE/ASSA San Diego CA, January 2020, Finnish Economic Association, February 2020, and the workshop at the University of Vilnius, December 2019. Data availability The anonymized data without student or school identifying information are available from the authors. We plan to make it publicly available to the Finnish Social Science Archive. References Abadie, A., and G. W. Imbens. 2006. Large sample properties of matching estimators for average treatment effects. Econometrica 74 (1): 235–67. doi: 10.1111/j.1468-0262.2006.00655.x. ———. 2016. Matching on the estimated propensity score. Econometrica 84 (2): 781–807. doi: 10.3982/ECTA11293. Amagir, A., W. Groot, H. Maassen van den Brink, and A. Wilschut. 2018. A review of financial-literacy education pro- grams for children and adolescents. Citizenship, Social and Economics Education 17 (1): 56–80. doi: 10.1177/2047173417719555. Anderson, M. L. 2008. Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects. Journal of the American Statistical Association 103 (484): 1481–95. doi: 10.1198/016214508000000841. Batty, M., J. M. Collins, and E. Odders‐White. 2015. Experimental evidence on the effects of financial education on ele- mentary school students’ knowledge, behavior, and attitudes. Journal of Consumer Affairs 49 (1): 69–96. doi: 10.1111/ joca.12058. Brown, S., and K. Taylor. 2016. Early influences on saving behaviour: Analysis of British panel data. Journal of Banking & Finance 62: 1–14. doi: 10.1016/j.jbankfin.2015.09.011. Caliendo, M., and S. Kopeinig. 2008. Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys 22 (1): 31–72. doi: 10.1111/j.1467-6419.2007.00527.x. Carpena, F., S. Cole, J. Shapiro, and B. Zia. 2019. The ABCs of financial education: Experimental evidence on attitudes, behavior, and cognitive biases. Management Science 65 (1): 346–69. doi: 10.1287/mnsc.2017.2819. Collins, J. M., and E. Odders-White. 2015. A framework for developing and testing financial capability education pro- grams targeted to elementary schools. Journal of Economic Education 46 (1): 105–20. doi: 10.1080/00220485.2014.976325. Ding, W., and S. F. Lehrer. 2010. Estimating treatment effects from contaminated multiperiod education experiments: The dynamic impacts of class size reductions. The Review of Economics and Statistics 92 (1): 31–42. doi: 10.1162/ rest.2009.11453. Drexler, A., G. Fischer, and A. Schoar. 2014. Keeping it simple: Financial literacy and rules of thumb. American Economic Journal: Applied Economics 6 (2): 1–31. doi: 10.1257/app.6.2.1. Fernandes, D., J. G. Lynch, Jr., and R. G. Netemeyer. 2014. Financial literacy, financial education, and downstream finan- cial behaviors. Management Science 60 (8): 1861–83. doi: 10.1287/mnsc.2013.1849. Frisancho, V. 2020. The impact of financial education for youth. Economics of Education Review 78: 101918. doi: 10.1016/j. econedurev.2019.101918. Gremmen, H., and J. Potters. 1997. Assessing the efficacy of gaming in economic education. Journal of Economic Education 28 (4): 291–303. doi: 10.1080/00220489709597934. EFFECTS OF GAME-BASED FINANCIAL EDUCATION 125 Hansen, W. L., M. K. Salemi, and J. J. Siegfried. 2002. Use it or lose it: Teaching literacy in the economics principles course. American Economic Review 92 (2): 463–72. doi: 10.1257/000282802320191813. Iterbeke, K., K. De Witte, K. Declercq, and W. Schelfhout. 2020. The effect of ability matching and differentiated instruc- tion in financial literacy education. Evidence from two randomised control trials. Economics of Education Review 78: 101949. doi: 10.1016/j.econedurev.2019.101949. Kaiser, T., A. Lusardi, L. Menkhoff, and C. Urban. In press. Financial education affects financial knowledge and down- stream behaviors. Journal of Financial Economics. doi: 10.1016/j.jfineco.2021.09.022. Kaiser, T., and L. Menkhoff. 2017. Does financial education impact financial literacy and financial behavior, and if so. The World Bank Economic Review 31 (3): 611–30. doi: 10.1093/wber/lhx018. ———. 2018. Active learning fosters financial behavior: Experimental evidence. Discussion papers of DIW Berlin 1743. Berlin, Germany: German Institute for Economic Research. ———. 2020. Financial education in schools: A meta-analysis of experimental studies. Economics of Education Review 78: 101930. doi: 10.1016/j.econedurev.2019.101930. Kalmi, P. 2018a. The effects of financial education: Evidence from Finnish lower secondary schools. Economic Notes 47 (2–3): 353–86. doi: 10.1111/ecno.12114. ———. 2018b. The effects of Me and My City on primary school students’ financial knowledge and behavior. Paper pre- sented at the Cherry Blossom Financial Education Institute Conference, Washington DC, April 12–13. Kalmi, P., M. Maliranta, and S. Alanko. 2019. Kohti tiiviimpää vuorovaikutusta koulujen taloustiedon opetuksen ja taloustieteen välillä (Toward a tighter interaction between economic knowledge taught in schools and economics). Kansantaloudellinen Aikakauskirja 115 (4): 604–19. Kneppers, L., M. Elshout-Mohr, C. van Boxtel, and B. van Hout-Wolters. 2007. Conceptual learning in relation to near and far transfer in the secondary school subject of economics. European Journal of Psychology of Education 22 (2): 115–29. doi: 10.1007/BF03173517. Lonka, K., J. Makkonen, M. Berg, M. Talvio, E. Maksniemi, M. Kruskopf, H. Lammassaari, L. Hietajärvi, and S. K. Westling. 2018. Phenomenal learning from Finland. Helsinki, Finland: Edita Publishing. Lührmann, M., M. Serra-Garcia, and J. Winter. 2015. Teaching teenagers in finance: Does it work? Journal of Banking & Finance 54: 160–74. doi: 10.1016/j.jbankfin.2014.11.009. Lusardi, A., and O. S. Mitchell. 2014. The economic importance of financial literacy: Theory and evidence. Journal of Economic Literature 52 (1): 5–44. doi: 10.1257/jel.52.1.5. Lusardi, A., A. Samek, A. Kapteyn, L. Glinert, A. Hung, and A. Heinberg. 2017. Visual tools and narratives: New ways to improve financial literacy. Journal of Pension Economics and Finance 16 (3): 297–323. doi: 10.1017/S1474747215000323. Mandell, L., and L. S. Klein. 2009. The impact of financial literacy education on subsequent financial behavior. Journal of Financial Counseling and Planning 20 (1): 15–24. Maynard, N. W., P. Mehta, J. Parker, and J. Steinberg. 2012. Can games build financial capability? Financial entertainment: A research overview. In Financial Literacy Center Working Paper WR-963-SSA. Santa Monica, CA: RAND Corporation. http://www.rand.org/pubs/working_papers/WR963.html/. Organisation for Economic Cooperation and Development (OECD). 2020. PISA 2018 results (Volume IV): Are students smart about money? PowerPoint. Paris: OECD. Siegfried, J., A. Krueger, S. Collins, R. Frank, K. McGoldrick, R. MacDonald, and G. Vredeveld. 2010. Voluntary national content standards in economics. New York: Council for Economic Education. Skimmyhorn, W. 2016. Assessing financial education: Evidence from boot camp. American Economic Journal: Economic Policy 8 (2): 322–43. doi: 10.1257/pol.20140283. Sutcliffe, M. 2002. Simulations, games and role-play. In The handbook for economics lecturers: Teaching, ed. P. Davies, 17– 20. Bristol, England: Economics LTSN. Walstad, W. B., K. Rebeck, and R. A. MacDonald. 2010. The effects of financial education on the financial knowledge of high school students. Journal of Consumer Affairs 44 (2): 336–57. doi: 10.1111/j.1745-6606.2010.01172.x. Walstad, W. B., and J. C. Soper. 1988. A report card on the economic literacy of U.S. high school students. American Economic Review 78 (2): 251–56. Willis, L. E. 2008. Against financial-literacy education. Iowa Law Review 94 (1): 197–285.