EVALUATION OF A SUMMER COURSE IN ENTREPRENEURSHIP FOR DISADVANTAGED HIGH SCHOOL STUDENTS

_ Abstract This paper overviews a pre-college entrepreneurial education program and examines its impact on entrepreneurial intention. The program, entitled the Think Like an Entrepreneur Summer Academy, aims at disadvantaged and underrepresented high school students and is developed by a public research university in the northeast USA. We analyzed the data from 154 graduates of the program and found a significant increase in entrepreneurial intention in the posttest compared to the pretest. The study adds to the literature on exploring and identifying the characteristics of pre-college entrepreneurial education interventions that help increase entrepreneurial intent and provides suggestions for future research, including comparing dropouts to graduates and exploring the interplay between entrepreneurial intention and entrepreneurial mindset.


Introduction
Entrepreneurship education offers the potential to create wealth by unlocking the entrepreneurial potential of students.But if it is restricted to postsecondary students, it will fail to help those who are too young for college and/or who do not plan to attend college.This creates a niche for entrepreneurship education aimed at younger students.
The remainder of the paper is organized as follows.We begin by reviewing prior research on entrepreneurial intention and entrepreneurship education followed by an overview a pre-college entrepreneurship education program entitled the Think Like an Entrepreneur Summer Academy (TLAE).We then discuss the study objectives, measures, hypotheses, research methodology, data collection, and the results of data analysis.The paper ends with limitations, conclusions, and suggestions for future research.

Entrepreneurial intention
Entrepreneurial intention is defined as "the conscious state of mind that directs personal attention, experience, and behavior toward planned entrepreneurial behavior" (Obschonka et al., 2010, p. 64).More than one thousand published papers have studied entrepreneurial intention (Kaffka & Krueger, 2018).The theory of planned behavior asserts that behavioral intentions are one of the prime determinants of future behavior (Ajzen, 1991), as does the entrepreneurial event model (Shapero & Sokol, 1982).These theories are complementary and have guided most previous research on entrepreneurial intention (Schlagel & Koenig, 2014).
Entrepreneurial intention is important because of its relationship with entrepreneurial mindset.One of the more concise definitions of entrepreneurial mindset is "the inclination to discover, evaluate, and exploit opportunities" (Bosman & Fernhaber, 2018, p.13), but no single definition has dominated the literature (Larsen, 2022).Entrepreneurial mindset is modeled as an antecedent of entrepreneurial intention (Cui & Bell, 2022); consequently entrepreneurship educators are encouraged to focus on entrepreneurial mindset rather than on entrepreneurial intention (Kaffka & Krueger, 2018).An entrepreneurial mindset can influence entrepreneurial intention, but entrepreneurial intention can also either encourage or discourage the development of an entrepreneurial mindset as discussed in the following paragraphs.
Entrepreneurial intention also affects the cognitive components of the entrepreneurial mindset because it influences how people evaluate entrepreneurial opportunities.Specifically, Lynch and Corbett (2021) propose that an entrepreneur cycles between elaborating and implementing mindsets.The elaborating mindset is about assessing the desirability and feasibility of engaging in an entrepreneurial venture.Desirability and feasibility assessments are not entirely rational.They are partially influenced by the emotions that a decision-maker attaches to the outcomes of each choice (Slovic et al., 2005).If a low level of entrepreneurial intention is due to a negative emotional reaction to the thought of being an entrepreneur, the elaborating mindset will never be fully engaged and the decision to forego entrepreneurship will be made regardless of the potential benefits of becoming an entrepreneur.
To sum up, entrepreneurial intention is important because it predicts entrepreneurial behavior (Joensuu-Salo et al., 2020) and because it has a complex relationship with entrepreneurial mindset.

Impact of entrepreneurship education on entrepreneurial intention
Most evaluations of the impact of entrepreneurship education on entrepreneurial intention have taken place at the university level.Martinez-Grigorio et al. (2021) conducted a meta-analysis of the effects of entrepreneurship education on a variety of dependent variables.Almost every study that they analyzed was at the tertiary, or university, level.Entrepreneurial intention was one of the few dependent variables for which education exerted a significantly positive effect.More recent studies (Borges et al., 2021;Duong, 2022;Hoang et al., 2021;Martins et al., 2022;Soomro & Shah, 2022) yield similar results.
However, Brüne and Lutz (2020) found that, at the pre-university level, entrepreneurship education exerted a positive effect on entrepreneurial intention in only three of the eight studies that they reviewed.Those three studies with positive results were situated in the United Kingdom (Athayde, 2009) and Spain (Barba- Sánchez & Atienza-Sahuquillo, 2016;Sanchez, 2013).The other five studies in their review were in Netherlands (Huber et al., 2014), Portugal (Marques et al., 2012), Switzerland (Volery et al., 2013), and the United Kingdom (do Paço et al., 2015;Thompson & Kwong, 2016).
Why would entrepreneurship at the pre-college level be less effective than collegiate entrepreneurship education in boosting entrepreneurial intention?It is an ideal of entrepreneurship education to give students a deep understanding of how entrepreneurs think, through starting their own businesses or at least through simulations of the experience (Neck & Greene, 2011).Learning through practice is one of several principles of adult education that have permeated entrepreneurship education (Neck & Corbett, 2018), but adult education may become less effective the farther that students are from adulthood.
Individualism is a positive predictor of entrepreneurial intention (Li n et al., 2016) and the United States is generally considered to be one of the world's most individualistic countries (Hamamura, 2012).There is research on entrepreneurship education at the pre-college level in the United States (e.g., DeBerg, 2000;Fitzgerald, 1999;Godsey & Sebora, 2010) but none tested its impacts on entrepreneurial intention.Therefore, it is possible that pre-college entrepreneurship education positively impacts entrepreneurial intention in the United States even though the evidence from other countries is not entirely supportive.

The 'Think Like an Entrepreneur' Summer Academy
The 'Think Like an Entrepreneur' (TLAE) Summer Academy is a summer program developed by a public research university in the Northeast USA.It is targeted at underprivileged and under-represented or otherwise marginalized high school students.The course was first offered in the summer of 2017 as part of a program to create pathways to success for local high school students.Its success in achieving that mission has resulted in its continuation, with attendant growth in registration from fewer than 50 students in the first year to more than 100 in its fifth offering, with registrants from more than 50 schools including two global locales outside the USA.
The program is marketed via email and online descriptors to principals, administrators, guidance counselors, and business program educators in the New Jersey and Philadelphia regions.Secondary school administrators also post application information to their social media networks, which has helped the program gain traction among parents and high school students living outside the region.Individual applications are completed through an online site that requests demographic and location information, with priority given to students from underprivileged and underrepresented populations, without requiring need-based evidence.
The course is sponsored, with direct costs underwritten by an international philanthropic organization.It is provided free of charge to all accepted participants.Students who successfully complete the course earn three semester hours of college-level credit which equates to one university-level course.Assurance of learning includes multiple assessments including quizzes, individual and team assignments, and online discussions.Student final grades in the class are determined on a "pass/fail" basis, with a C+-level minimum point score required to pass and earn course credit.
The course is designed to be very similar to an entry-level University entrepreneurship course that does not require any academic prerequisites.The course goals include help students develop entrepreneurial mindsets; help them understand business models and basic business skills; teach them to transform ideas into feasible businesses; introduce them to the excitement of working with a team toward a common goal; increase their awareness of social issues; and provide first-generation college students with a realistic preview of college life.The course description as it appears in the syllabus is as follows: "Entrepreneurship is one of the major drivers for economic change and employment.However, the path to launch a successful entrepreneurial venture and develop an opportunity driven mindset requires a systematic approach.The TLAE Entrepreneurship and Innovation course introduces students to understanding and experiencing the qualities and thinking of successful entrepreneurs and innovators and describes possible entrepreneurial career paths available.The course aims to provide students with basic perspectives to better understand and establish a business or social venture and to introduce students to unique and interesting problems facing entrepreneurs in starting and operating new ventures." The course is managed by an interdisciplinary team of faculty who delivers lectures, guide hands-on learning activities, and orchestrates guest speakers.Additional teachers, speakers, and mentors are pulled from the university's network of entrepreneurially informed senior-level students and real-world entrepreneurs who are advisory council members.The course has been delivered on-ground and in-person at its university location in southern New Jersey, and through synchronous online modality during the COVID-restricted summers of 2020 and 2021.
Instructors cover the full range of entrepreneurial activities, including problem and opportunity recognition, design thinking, market definition, rapid prototyping, the Business Model Canvas, finance and resource modeling, intellectual property, the power of networking, the advantages of accelerators, pivoting, pitch development, and product development.In its on-ground format, the class meets together in-person, four days each week for two weeks, for five hours each day.For the on-ground delivery, the Program provides transportation for certain underprivileged communities to ensure their ability to participate.In its online format, the class meets synchronously twice a week for two-and-a-half hours.
Teams are assigned by the instructors at the start of the course with the intention of seeking diversity of gender, race, academic institution, and socioeconomic background.This process also helps to prepare students for similar diversity they will likely experience when attending university.Creating diversity is additionally important for team-based creative problem solving, which is an essential competency of the entrepreneurial mindset.All experiential exercises and assignments are conducted by teams to replicate the importance of teamwork in entrepreneurial settings.To further emphasize the importance of teams, grades for all work other than quizzes are collectively assigned to the team, and a mid-point peer review of team members is conducted.In the final week, teams present their ideas to a panel of judges with awards of certificates and prize bags.
In terms of entrepreneurial opportunity that can address global challenges, from 2019 onwards the Program decided to focus specifically on the United Nations Sustainable Development Goals (SDGs), which provide an excellent framework for teaching entrepreneurship concepts in a short period of time to students who don't have business or college backgrounds.Because the SDGs are validated problems, students spend more time on solution validation.The SDGs are also consistent with the host university's mission and the Program has identified complementary resources to teach them.Students are introduced to the SDGs in the first class, with a theme of "Our world needs you to think like an entrepreneur."Students learn concepts of sustainability and the attributes of the entrepreneurial mindset and shown examples of how entrepreneurs can support the SDGs.During the second class, a guest speaker from the United Nations lectures on the importance of the SDGs, which helps students appreciate the possibilities of genuinely deep impact and develops their empathetic perspective.

Measures
We adopted the six-item scale developed by Thompson (2009) to assess individual entrepreneurial intent and the six-item scale developed by Farmer et al. (2011) to assess entrepreneur identity aspiration.Although these two scales appear related, Vanevenhoven and Liguori (2017) suggest they represent separate constructs.Table 1 lists the items associated with the two scales.A five-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree) was used to measure all the items.

Table 1
The two-construct measurement model

Entrepreneurial Intent (EI)
EI-1: Intent to set up a company in the future EI-2: Never search for business start-up opportunities (R) EI-3: Are saving money to start a business EI-4: Read books on how to start a company EI-5: Have no plans to launch your own business (R) EI-6: Spend time learning about starting a firm Note: Items marked (R) are reverse coded in scale analysis.

Entrepreneur Identity Aspiration (EIA)
EIS-1: I often think about becoming an entrepreneur.EIS-2: I would like to see myself as an entrepreneur.EIS-3: Becoming an entrepreneur would be an important part of who I am.EIS-4: When I think about it, the term "entrepreneur" would fit me well.EIS-5: I am always thinking about becoming an entrepreneur.EIS-6: It is important for me to express my entrepreneurial aspirations.

Hypotheses
We formulated two hypotheses to examine the effectiveness of the TLAE program: H1: The TLAE program for disadvantaged high school students has a positive influence on the students' entrepreneurial intentions.

H2:
The TLAE program for disadvantaged high school students has a positive influence on the students' entrepreneur identity aspiration.

Single-Group Pretest-Posttest Design
We employed the single-group pretest-posttest design to examine the effectiveness of the program by comparing the scale scores before and after attending the program.This design is considered superior to cross-sectional designs (Seifert et al., 2010) and continues to be utilized in educational research (e.g., Flasch et al., 2017) including entrepreneurship education research (e.g., Plachkinova & Pittz, 2021;Yamakawa, et al., 2016).
The concerns about the pretest-posttest design include its lack of a control group (de Anda, 2007) and potential problems associated with internal validity such as history and regression to the mean (Knapp, 2016;Marsden & Torgerson, 2012).These concerns are not pertinent to our study.We have no reason to believe that, on average, the entrepreneurial intentions of a control group would have increased during the summer.Furthermore, TLAE was a voluntary program that was not easy to complete.Thus, it is reasonable to assume that participants possessed a relatively high level of entrepreneurial intentions before the program started, so that the pretest did not underestimate the true levels of that construct.We contend that this testing is appropriate in the given context and the testing results can be informative.

Data collection
We collected data through pre-and post-course student surveys.The surveys included all12 questions that comprise both scales discussed in the previous section.The post-course survey also included an open-ended question: "What was your biggest takeaway from the TLAE summer program?"The data were collected online using the Qualtrics survey tool.Completion of the surveys is a graded work item in the course (5% in total, combined) to promote participation.The pre-course survey was opened three days before the start of the course and closed before the first day of the course.The Qualtrics survey link was emailed to all registered students before the course commencement, while the post-course survey link was posted on the course learning management system site.The post-course survey was opened on the last day of the course and closed two days after course completion.The data used in this study were collected as part of the 2020 and 2021 online deliveries of the course.One co-author matched the responses of the pre-and post-course surveys by the names of the students, deleted the student names, and then transferred the data to two other coauthors who performed the statistical analyses.

Data analysis
Before assessing the hypotheses, we performed confirmatory factor analysis (CFA) to validate the two-scale measurement model adopted from previous studies, using IBM AMOS V29 and employing Maximum Likelihood estimation.The factor structure was assessed for pre-and post-course survey data, respectively.Following prior practices (e.g., Agarwal & Prasad, 1999;Babin & Boles, 1998), the average value of all remaining items on each scale was then used to represent that scale.Paired t-tests were employed to assess the mean difference of each scale in the pre-test and the post-test.
To supplement the results of quantitative data analysis, we further analyzed students' written responses to the open-ended question in the post-course survey: "What was your biggest takeaway from the TLAE summer program?"

Results
One hundred fifty-four students completed the pre-and post-course surveys in 2020 and 2021.The participants were between 15 and 18 years old, averaging 16.3 years.Slightly more than half of the participants (54%) were male.

Results of CFA
Table 2 presents the CFA results for the two-factor model adopted from the literature.The model chi-square values (87.99 for pre-and 124.29 for post-course data) have a p-value <0.01.The literature suggests that one limitation of using the chisquare test of model fit is that researchers may accept an inappropriate model in a small sample size and reject an appropriate model in a large sample size (Gatignon, 2010).Accordingly, we examined a set of commonly used measures of fit.As shown in Table 2, almost all of them fall within the acceptable ranges, indicating the overall fitness of the model to the data.
Table 3 presents the factor loadings and results for convergent and discriminant validity measures.All factor loadings are 0.5 or higher and considered practically significant for the sample size (Hair et al., 1995, p. 385).Convergent validity was assessed using composite reliability (CR) and average variance extracted (AVE).According to Fornell and Larcker (1981), an AVE below 0.5 could be considered if CR is above 0.7.Discriminant validity was assessed using the Heterotrait-Monotrait ratio of correlations (HTMT) with a threshold of 0.9, as suggested by Henseker et al. (2015).It is evident from Table 3 that the two-factor measure model has achieved good convergent and discriminant validity as per the stated criteria.In sum, the results of CFA show a reasonably good fit of the measurement model to our data and suggest that the students could distinguish clearly among the two constructs under study. EI-2 was removed due to its low loading (<0.4).

Results of the Hypothesis Tests
Table 4presents the results of paired sample t-tests.There was a positive change in Entrepreneurial Intent.The mean of post-course survey (μ=3.12,s=0.84) was significantly higher than the mean of pre-course survey (μ=2.93,s=0.75), with t=3.11& p=0.001.The result supports H1 that the TALE program has a positive influence on students' entrepreneurial intention.Entrepreneur Identity Aspiration did not change significantly after the course (μ=3.63,s=0.96) as compared to prior to the course (μ=3.72,s=0.92), with t=-1.41&p=0.08.Thus, H2 was not supported.To further evaluate the effectiveness of the TALE program, we summarized and analyzed students' written responses to the open answer question included in the postcourse survey: "What was your biggest take away from the TLAE summer program?" (see Table 5).Students have noted appreciation for learning about business and values of planning and creating a business.One student comment sums up the Program's entire goal with one simple statement in his email to one of the course instructors: "I was beginning to believe in my ability to really change the world."

Table 5.
Selected quotes from the open-ended posttest question I took away how to appeal to investors and market my ideas as a business that will make good returns even if that was not originally my intent My big take away is that there was always a part of me that wanted to change the world in some type of way whether if it was creating something or just helping people.I learned a lot about the value of creating your own business and how if you really work hard, it could turn out to be quite exciting and profitable.I learned that I should definitely continue learning about business because it is what I want to do.Always look for new ways to improve as an entrepreneur.My time will come.I learned a lot about how much work it takes to even form a plan for a business let alone starting one.I know I will use these steps and skills in the future I am now more open-minded about business ideas and opportunities I hope to carry [these skills] all throughout college and life, maybe as a business owner myself.Using this knowledge, I can keep on thinking of different ways to help solve these problems by creating a unique company.Before entrepreneurship seemed distant and merely possible, while now it is something much more familiar, and something I know I can one day do myself.I will use these [ideas] to continue to pursue my entrepreneurial aspirations.

Discussion
We observed an increase in Individual Entrepreneurial Intent but not in Entrepreneur Identity Aspiration.As we noted above when we introduced the dependent variables, this outcome supports a direct impact of this program on entrepreneurial intent.Had both dependent variables increased, a halo effect may have been the cause.The relatively modest size of the increase should not be viewed as an indictment of the Think Like an Entrepreneur Summer Academy.The lure of three semester hours of college credit would probably be insufficient on its own to induce a teenager to spend summer learning about entrepreneurship, so the students must have had a strong prior interest in entrepreneurship or else they would not have expended the necessary efforts to complete the program.
As we previously discussed, one criticism of the single group pretest-posttest design is regression to the mean, which occurs when students are selected for an educational intervention due to their poor performance with respect to a targeted outcome.The opposite situation applies in this instance.Students self-selected into this program based on their high level of interest in becoming an entrepreneur, therefore an improvement in any entrepreneurial attitude in the posttest compared to the pretest can confidently be attributed to the effects of the Think Like an Entrepreneur Summer Academy on its participants.

Limitations
We only examined entrepreneurial intent and entrepreneur identity aspiration.There are many other outcomes of entrepreneurship education that are worthy of study (Brüne & Lutz, 2020).We had no control group so we cannot be certain that the same results would not have been observed in a control group, although there appears to be no theory-based explanation as to why that would occur.External validity is problematic because we only evaluated students who were enrolled in one program over two summers within one country, therefore we cannot claim that our results conclusively prove the benefits of entrepreneurship education at pre-college level.
Another limitation was our exclusive use of an explicit measure of entrepreneurial intention.Mai and Dickel (2021) make a persuasive case that entrepreneurial intention can best be understood through a combination of explicit and implicit measures.Implicit measures, which are commonly used in psychology research and increasingly popular in business research, calculate the speed with which respondents pair a label such as "self-employment" with an attribute such as "me," and more rapid pairings indicate stronger implicit beliefs.

Conclusions and suggestions
This research adds to our understanding of the effects of entrepreneurship education because it is conducted in the United States, because participants were high school students, and because entrepreneurial intentions were measured at the beginning and the end of the program.We find an increase in entrepreneurial intention in the posttest compared to the pretest, which shows the success of this program.One thing we do not know is why The Think Like an Entrepreneur Summer Academy succeeded.We gathered data on what students liked and disliked about the program, but reactions are generally considered to constitute poor evaluative criteria because they do not measure learning, behaviors, or outcomes (Arthur et al., 2003).
Future research directions abound.For example, the original plan was to offer the program onsite but due to COVID-19 it was switched to synchronous online instruction.In future post-pandemic summers, we will replicate our research design.We may find that the richer educational environment of face-to-face instruction leads to greater increases in entrepreneurial intention, or it may turn out that the inconvenience of leaving the home to learn about entrepreneurship reduces student attitudes towards the program and towards entrepreneurship as a career choice.
We also need to study dropouts.Not everybody who started the program completed it.Perhaps dropouts had lower levels of entrepreneurial intention than graduates, in which case they lacked the motivation to do the necessary work.
Alternatively, dropouts may have higher levels of entrepreneurial intention than graduates and they may have left the Think Like an Entrepreneur Summer Academy so they could spend more time being entrepreneurs and less time learning how to be entrepreneurs.
We have argued that a single group pretest-posttest design is adequate to assess entrepreneurship education that is delivered during the summertime to voluntary participants.However, if there was some way to include a control group that would strengthen our conclusions.It would also be interesting to know what activities precollege students do during the summer that relate to entrepreneurship if they are not participating in structured educational activities, and how these affect their intentions to become entrepreneurs.
It would be informative to follow up with the program participants in the future, to determine if they became entrepreneurs.Schoon and Duckworth (2012) found that entrepreneurial intention in the teenage years predicted entrepreneurial behavior later in life.Finally, it would be intriguing to study the effects that entrepreneurial intention and entrepreneurial mindset have on each other.We have suggested that entrepreneurial intentions facilitate or inhibit entrepreneurial mindsets and that entrepreneurial mindsets enhance or diminish entrepreneurial intentions.If we are correct, then entrepreneurship educators should aim to improve both entrepreneurial intention and entrepreneurial mindset.This could be evaluated through a repeated measures design in which entrepreneurial intention and entrepreneurial mindset were measured at the beginning, middle, and end of an intervention.But it will have to await a widely accepted measure of entrepreneurial mindset, which cannot happen until researchers coalesce around a common definition of that construct.
We began by noting that most studies of entrepreneurial education have examined college-level programs and that the evidence supporting an effect of entrepreneurial education on entrepreneurial intention is more persuasive at the college level than at the pre-college level.Pre-college entrepreneurship education is not doomed to failure, as minorities of studies have observed positive impacts.More studies in more countries will help to identify the characteristics of pre-college entrepreneurship education interventions that increase entrepreneurial intent.

Table 2
Results of model fit

Table 3
Factor loading and validity of the measurement model