Office of Strategy Management
Phase III: Academic Management Audit
Phase III: Academic Management Audit (June 2016 – April 2017) - Click to read full report (PDF)
The Academic Management Audit represents the third and final phase of the Educational and Operational Efficiency Audit. Programs included in this review are:
- General Education Programs
- Instructional Technology
- Exceptional Student Education (ESE) Programs
- English Language Learner (ELL) Programs
- Career and Technical Education (CTE) Programs
The Phase III work began in September 2016, with issuance of a comprehensive data request. The Gibson review team collected and analyzed data, conducted numerous interviews, focus groups, and school site visits, and then synthesized all of this information into the findings and recommendations presented in this Phase III final report. Below is a more detailed description of each of these activities.
To conduct a comprehensive Educational and Operational Efficiency Audit of HCPS, the review team used a variety of data collection and analysis approaches. This comprehensive review of HCPS’ instructional areas included the following data collection approaches.Existing HCPS Data
To provide proper context for the review, Gibson requested from the HCPS a broad spectrum of data and documents related to the instructional areas under review. The purpose of this data request and subsequent analyses was to gain a deeper understanding of HCPS programmatic operations and provide background and context for the review. In addition, these data and documents were utilized to help formulate questions for the interviews and focus group sessions held with district administrators, department heads and staff, school administrators and staff, and teachers.Interviews with District Staff
To ensure that the review team had a complete and thorough understanding of district programs, processes, and procedures, interviews with key staff were conducted. Interviews included district leadership, department heads and staff, and school administrators and support staff, among others.School Site Visits
A sample of 35 HCPS schools was selected for site visits based on school type, geographic location within the district, and in some cases, programmatic information. The primary objectives of the school visits were to interview school leaders, and conduct classroom walkthroughs to observe teacher practices, student participation, and classroom design.Focus Group Sessions
Focus groups are an effective way of obtaining more in‐depth information from staff than a one‐on‐one formal interview or other data collection instruments. In addition, the dynamics of a focus group often stimulate the expression of ideas that might otherwise go unstated. The project team conducted focus group sessions with varying groups of stakeholders (e.g., principals, teachers, operational area leads, departmental and school staff).Peer, State and National Comparisons
Gibson used the most recent state academic, expenditure and staffing reports to compare HCPS to state and peer averages. These reports are available annually; the most recent reports available at the time of this study contained actual expenditure and staffing data through the 2014‐15 school year. Unaudited actual expenditure data for 2015‐16 was also included. Where applicable, Gibson also applied national or other benchmarks for comparison to HCPS.
During the assessment phase of this project, data for each instructional area was reviewed and analyzed to assess student enrollment trends, organizational alignment and staffing, resource allocation, student performance trends, major compliance requirements, and programmatic design and implementation.Interview and Focus Group Data
Qualitative interview and focus group data were analyzed by functional area leads conducting the focus group sessions to determine common themes across the various stakeholder groups (e.g., district administration, school leaders and staff, department heads and staff). Other sources of input (e.g., observations, district data, and industry best practices) were also included in analyses.
HCPS is the eighth largest school system in the U.S. with more than 207,000 students enrolled in 2016‐17. Student enrollment has increased nearly 7 percent in the past five years, primarily due to significant increases in charter school enrollment. Excluding charter schools, total K12 enrollment is approximately 190,000, and has grown less than 2 percent over the past five years – an average of 0.33 percent annually.
Student performance in HCPS over the past few years has been relatively flat, and scores on the Florida Standards Assessment (FSA) exams have been consistently below or at the Florida state average. HCPS’ passing rate (Level 3 and above) for English/Language Arts (ELA) is below the state average for all grade levels, and the performance gap widens as students move from elementary to high school (see Figure 1).
Figure 1. Florida Standards Assessments, ELA by Grade Groupings, Spring 2016 Compared to Spring 2015
Student performance in Math is higher overall; however, HCPS performance lags significantly behind the state average in elementary school but rises to the state average in middle school.
Figure 2. Florida Standards Assessments, All Math by Grade Groupings, Spring 2016 Compared to Spring 2015
By the time students are in high school, HCPS students outperform the state average on the Algebra 1, Geometry, and Algebra 2 End of Course (EOC) exams.
Figure 3. End of Course (EOC) Exams, Spring 2016 District Name
One of the most important aspects of a school district’s mission is to provide a high quality education that sets high standards for all students, including those in special populations. Mirroring national statistics, student performance for HCPS’ economically disadvantaged, SWD, and ELL students is significantly below that of their general education peers. In 2015‐16, the performance gap between economically disadvantaged and non‐economically disadvantaged students in Math and ELA is 34.8 and 34.3 percentage points, respectively. From 2014‐15 to 2015‐16, economically disadvantaged student performance in Math increased slightly (from 36 to 36.8 percent), while their non‐economically disadvantaged peer’s performance decreased (from 73.3 to 71.6 percent)—narrowing the gap by 2.5 percentage points. The performance gap narrowed slightly in ELA as well (1.2 percentage points), although ELA performance for all students declined from 2014‐15 to 2015‐16.
Not surprisingly, the achievement gap on the FSA is wider for HCPS special education students – 42.9 percentage points in Math and 38.9 percentage points in ELA. From 2014‐15 to 2015‐16, SWD performance decreased in both ELA and Math, while the performance of their non‐disabled peers increased slightly. When compared to the state average, HCPS SWD students consistently perform below their disabled peers at every grade level in both Math and ELA. The passing rate for SWD students (all grades) on the Algebra 1 EOC exam is 18.8 percent, which is 41.5 percentage points below the HCPS nondisabled peer average of 60.3 percent.
ELL student performance on the FSA shows lower but improving performance. Over the past two years, ELL passing rates in ELA increased from 12.7 to 13 percent, while non‐ELL student passing rates slightly declined from 55 to 54.5 percent. The achievement gap in Math is much narrower: ELL passing rates increased from 43.4 to 45.8 percent, while non‐ELL student passing rates remained flat at 53.9 percent.
At HCPS, CTE concentrators (i.e., students who earn three or more credits in a single CTE program) experience a graduation rate that, on average, is 19.5 percentage points higher than the overall district average.
It is important to acknowledge that the Phase III work assessing HCPS academic programs was conducted during a time of significant change in the district. Major district‐level organizational changes had occurred or were still being implemented; area superintendent responsibilities were just changed to include academic oversight; major cost reductions were being implemented; student data analysis tools were being expanded; and other academic programming changes were occurring. The impact of these recent changes was obviously not fully realized during the Phase III work. The review team attempted to acknowledge recent efforts, provide recommendations that may help this transition be more successful, and provide additional recommendations to improve student achievement at HCPS.
Research suggests the factors that most improve student learning are excellent teachers, high expectations, a rigorous curriculum aligned with standards, and instruction that is adjusted to meet specific student needs based on ongoing formative assessment of student progress against standards. Most of the recommendations in this report are focused on those areas specifically aimed at improving student performance in HCPS. With the exception of one recommendation to invest more in instructional technology, the review team has concluded that nearly all of the recommendations in this Phase III report can be implemented using the district’s existing resources.
First, the review team identified several areas of best practice for which the district should be commended.
- HCPS provides early screening and intervention programs to identify and serve special education students from birth to age five. HCPS has partnered with other community‐based organizations to provide a developmental screening program to identify children who may have delays in speech and language, hearing, vision, cognitive development, motor skills, or have other social‐emotional challenges, so they can be referred for further evaluation and/or assistance. In addition, HCPS' Early Childhood ESE program has more than doubled the percentage of students served inside the general education classroom in the past three years (from 24 to 51 percent), the least restrictive setting. These two initiatives are significant, as research shows that 1) high quality early intervention is likely to lead to improved outcomes for children, which is less costly over time, and 2) young children who spend most of their day alongside their non‐disabled peers benefit from having positive role models for all of the key areas of development, not to mention the numerous benefits non‐disabled children receive.
- HCPS utilizes an on‐line comprehensive assessment system, SchoolCity, to develop, score, and report student assessment data. SchoolCity allows teachers and administrators to create formative, interim, and diagnostic assessments that are aligned to the Florida state standards. The easy‐to‐use scoring process and robust reporting capabilities allows for instructional staff to review results that are timely and insightful. SchoolCity is popular and widely used throughout the district.
- HCPS has forged relationships and garnered strong support from the community, evidenced by numerous partnerships with community‐based organizations, local businesses, and parent involvement programs. Leveraging the support of outside partners to the district is an excellent mechanism to improve the quality of education and at a lower cost. The ESE program has partnered with the Early Childhood Council of Hillsborough County (ECC) and the Florida Diagnostic and Learning Resources System (FDLRS) to provide an early childhood development screening program (described above), and the CTE program has established partnerships with the Outback Restaurant, Gregory Foundation, University of South Florida, Career Source, and many others. These partnerships give students opportunities for hands‐on and real‐world experiences, as well as the potential for internships and scholarships. The ELL program has a multifaceted parent outreach initiative to encourage parent involvement in their child’s schooling, as well as linkages to a variety of other community‐based programs and services. The Gifted and Talented program also has an innovative program to “spark interest” of Limited English Proficient (LEP) students through targeted learning experiences followed‐up with parent informational sessions and suggested activities to do at home.
The findings identified during the Phase III audit yielded several recommendations that are generally applicable to all of the district’s instructional programs included in the scope of this review.
- Increase the academic rigor of instructional programs and ensure that the curriculum is aligned to standards. One of the biggest barriers to improving student performance in HCPS is the lack of rigor in many of its instructional programs, particularly for struggling students. Rigor, in this case, is meant to describe instruction, school work, learning experiences, and educational expectations that are academically, intellectually, and personally challenging for all students. HCPS is working hard to address this challenge and is continuing the transition from teaching and learning practices that are program and textbook driven to a standards‐based education system. Although this process takes significant time and effort, adjustments to the districts implementation plan would help to facilitate a smoother and more efficient transition to the standards‐based instructional model. With respect to SWD students, the review team found that when compared to the state average, HCPS has a lower percentage of students placed in the general education setting and a higher percentage of students placed in a resource room setting. This raises the concern that SWD students do not have sufficient access to a rigorous curriculum taught by teachers with content area expertise, particularly in reading and math, where they are likely to need the most support. When looking at the district’s CTE program, the review team also found that many of the Programs of Study currently being offered were “hobby”‐type classes, as opposed to instructional programming that is more college and career‐ready. HCPS needs to better integrate and align technical education and academic standards while providing students with opportunities to connect to real‐world experiences in those areas where labor market trends are favorable.
- Negotiate with the Hillsborough County Classroom Teachers Association to allow for at least 60 to 90 minutes of collaborative planning time (or PLC meetings) per week. Research shows that collaborative planning time (CPT), when used well, is an important predictor of student achievement and one of the best uses of teacher time. The current teacher union contract specifies that “no more than one hour per month may be used for PLCs or other data gathering/planning intended to increase student achievement”. Best practice school districts support schools in providing 90 minutes of weekly collaborative planning time to allow teachers to meet in grade level and/or subject area teams, often with the support of an instructional coach or teacher leader, to review student formative assessment data, and discuss strategies to adjust instruction to better meet individual student needs.
- Provide more guidance and structured support to teachers on the effective use of formative assessments to drive student learning. The central idea behind the use of formative assessments is that evidence of student learning is used to adjust instruction to better meet student learning needs. The review team did not find sufficient or consistent evidence that teachers are systematically reviewing and analyzing formative assessment data with the specific goal of modifying teaching strategies for students that are struggling in a particular area or with a specific concept. This was especially true for SWD teachers. The need for additional support and guidance is exacerbated by the fact that collaborative planning time in most schools is limited to the monthly PLC meetings. The district could better support teachers during their PLC meetings by providing protocols to help structure the work of responding to short‐cycle formative assessments, and holding principals accountable for making sure this time is used well.
- Ensure that professional development is primarily job‐embedded and supported by lead teachers or instructional coaches. Research has shown that teacher quality is the single most important factor influencing student achievement. And despite investing significant funds over the years to improve teacher quality through professional development activities, HCPS’ professional development efforts can be characterized as fragmented, unfocused, and workshopbased. Faced with reduced funding levels due to the termination of the Bill and Melinda Gates Foundation grant, it is imperative that HCPS align its limited resources to ensure that all professional development activities revolve around how students are performing and what information and supports teachers need in order to continuously improve their instruction to meet student needs. Ensuring that professional development is job‐embedded and supported by an instructional coach or teacher leader is best practice. Together, with the implementation of 90 minutes of weekly collaborative planning time, HCPS could lay the foundation for student achievement to improve at a faster pace.
- Develop a district‐wide Instructional Technology Strategy and Plan. Without an instructional technology plan in place to serve as a guide for decision‐making as well as a tool to monitor and evaluate progress toward identified goals and objectives, instructional technology throughout the district is sparse, antiquated, not well‐integrated with curriculum and instruction, and is severely underfunded. This is evidenced by the fact that HCPS has more students per device than peer districts; the median age per device is 6 years old; and, 83 percent of devices are more than 5 years old, which is generally considered to be at the tail end of the replacement lifecycle. Further, 68 percent of devices are desktop computers and more than a third are located in computer labs, which limits their accessibility and use. Despite these statistics, HCPS has no immediate plans to make significant investments in instructional technology. A well‐integrated instructional technology program enhances and enriches learning opportunities for students, and increases the effectiveness of educators and support staff. Although a detailed needs assessment will need to be conducted, the review team estimates that if HCPS were to fully upgrade its inventory, as well as maintain an average replacement life cycle, then it will need to make a one‐time investment of $20 million, with annual investments of $10 million each year. Some of these costs can be mitigated with expansion of the Bring Your Own Device (BYOD) program.
- Chapter 1 – General Education Program
- Chapter 2 – Instructional Technology
- Chapter 3 – Exceptional Student Education Programs
- Chapter 4 – English Language Learner Programs
- Chapter 5 – Career and Technical Education Programs
Embedded in the remainder of this report are many other recommendations aimed at improving the overall efficiency and effectiveness of HCPS’ instructional programs.
Organization of Report
The remainder of this report is organized into the following chapters:
Entire Report (PDF)