Proactive data use…leading, not lagging

Student success is education’s purpose.  Student success, as measured by student results is how we evaluate our programs and services.  But we won’t succeed if we focus on evaluative data sets on a daily basis. 

Success comes from focusing on a series of process quality measures that are highly correlated with student success.  These measures called leading measures or process performance measures must be collected and analyzed on a regular basis at all levels of the organization.  They are constant feedback on how well we are succeeding with students.  These non-evaluative measures should guide our daily work and they help keep us on track for the end of program or end of school evaluation measures that we value.  They inform our process management and assure work quality.

The measures are called leading because they occur during the instructional process, not at the end of it like so many of the performance measures currently used in state accountability systems.  Leading measures monitor daily work based on research and best practices. 

Getting started

Creating leading measures is critical to long term success.  Here are some key steps to use to create impactful leading measures.

  1. Define student success.  Student success varies based on program purpose.  Alternative schools have a purpose different from career development programs.  These programs are different than academic, Family and Consumer Science, online, customized training, and workforce programs.  Defining what student success means for a program is a critical first step in the process of improving.  Too often state accountability systems define everything in terms of math and reading results.  Will necessary, these successes are not sufficient for long term life success.  

  2. Develop quantitative performance measures to determine the extent to which students are being successful within each program. 

  3. With the performance measures clearly defined, determine what influences student success.  These influencers must be separated into two sets, those that can be controlled and those that can’t. 

    The influencers beyond school control such as family circumstances, poverty, and prior academic achievement must be set aside.  Senge notes a learning disability entitled “The enemy is out there” in his book The Fifth Discipline.  He describes how blaming something or someone outside of our control for our current situation lets us avoid learning and growth.  It relieves us of responsibility for our circumstances.  Avoid this learning disability.  Not blaming student performance on influencers we can’t control means we must focus on those influencers within our control.  We then take responsibility for student success!

  4. The influencers that can be controlled are grouped and a series of major processes to control them are identified.  Examples of major processes include

    1. Governance

    2. Curriculum design and development

    3. Education delivery

    4. Enrollment, retention, and transition

    5. Finance and accounting

    6. Human resources management

    7. Information management

    8. Leadership

    9. Technology

  5. A set of key requirements must be established for each of the selected major processes.  These key requirements describe the conditions that needed to exist to properly control the influencers that would lead to student success.

  6. A process performance measure is then created for each major process or sub-process.  These leading measures are selected to provide feedback on the extent to which the procedures are being effective.  

    These measures are highly correlated with student success, the ultimate evaluation of our work.  Over time they may change as the cause and effect relationship between these leading measures and student performance become more clear.

This process of creating leading measures is critical to the long term acceptance and success of any process monitoring system.  Although it may be easy to simply implement another school district’s work, it will ultimately fail.  Faculty and staff must wrestle with the difficult discussions surrounding student success, influencers, major processes, key requirements, and leading measures before they can be ready to use these tools to create change.  Although some of the work may be the same, each organization will have its own unique response to these important areas.

Putting the measures to work

The leading measures are only effective if they are monitored and analyzed.  This is a daunting challenge when day-to-day activities demand so much attention in most districts.  To assure monitoring and analysis, major process owners and their teams are charged with the task of assuring leading measures are collected, analyzed, and reported.  Senior leadership reinforces the analysis of the leading measures by their administrators, faculty, and staff.  The combination of a series of cross-functional teams monitoring and analyzing results and the regular organizational structure supporting deep analysis dramatically improves the quality of analysis and attention. 

 An example of a leading measure is student engagement in standards-aligned curriculum work.  The research is very clear…engage students in active learning that is aligned to academic and industry standards and they will succeed in post high school pursuits.  One example of monitoring student engagement using a walk-through process is outlined in the book, The Three Minute Walk-Through (Downey, Steffy, English, Frase, Poston, 2004).  Administrators can monitor student engagement, curriculum alignment, and other critical factors on a bi-weekly basis in every classroom in the district.  This information is provided as professional, non-evaluative feedback to faculty so they can adjust instruction to assure full student engagement.  It is also collected at the building and the district level.  It drives decisions related to classroom practice, technology use and support, extra-help and intervention services, and professional development. 

In addition to organizational level analysis, division, building, and faculty level data is available to everyone.  Faculty receive this information on a bi-weekly basis.  They, of course, can also monitor student engagement directly daily.  

Continuous improvement…a challenge in new thinking

One of our greatest challenges is shifting thinking from the evaluation to feedback.  It is quite painful the first few times you look at leading measures that are far from perfect.  The tendency is to want to only report what works well.  Continuous improvement requires everyone to seek out failure and to be excited to find it.  Finding failure points and going about correcting the procedures that produce the failure is hard work.  Accepting shortcomings is difficult.  The extent to which a team can celebrate bad news and find it valuable is directly related to the extent to which continuous improvement can occur. 

Critical to the open, honest analysis process is the prohibition on this information being used in faculty or staff evaluations.  All leading measures are non-evaluative in nature.  This allows everyone to openly discuss the result, seek assistance freely, and take risks with creating new ways to improve results.

A core belief must be…anything we do today is inherently flawed and must be replaced with a better approach.  Anyone that looks back on their work with satisfaction is either a fool or hasn’t learned anything.  Ellen Glasgow once said, “No idea is so antiquated that it was not once modern.  No idea is so modern that it will not someday be antiquated.”  By focusing on improving leading measures instead of pushing the latest reform initiative, we deeply imbed continuous improvement practices. 

Staying focused on student success

Evaluate success by the number of students who succeed.  Assure student success by monitoring the quality of daily work using leading measures.  Use leading information to inform work improvement.  Evaluate improvements by the number of students who succeed.  By always coming back to student success as the only measure of success keeps the leading measures a useful feedback system for improving processes.  It also assures we don’t forget what is most important, student success.

(A)

Screen Shot 2018-10-24 at 10.04.20 PM.png

 

(B) Education Delivery

Key requirements for effective education delivery

  1. Education is delivered as planned with faculty having access to a course of study and instructional guides appropriate to their assignment.

  2. Faculty will have the ability to use the appropriate methodology and differentiated instruction to maximize student engagement.

  3. Efficient delivery of courses of study produce student performance

  4. Students receive high quality instruction consistent with the Butler Tech Learning Experience criteria including:

    1. Students are engaged in active learning

    2. Students are given extra help when needed

    3. Students are given regular on-going feedback on performance against content standards

    4. Students are prepared to take external assessment (i.e., industry credentials, ACT, Ohio graduation exam)

    5. Students are treated with respect

    6. Students find learning enjoyable and challenging

    7. Students are given confidence that they can succeed and believe effort will produce results

    8. Students have the necessary time to learn what is assessed

    9. Students know what is expected and what will be assessed

    10. Students recognize the importance of career and educational plans and career passports

    11. Students recognize the relevance of student organization membership and activities to their career and educational goals

    12. Students recognize the relevance of what they are learning to their career and educational goal.

    13. Students take personal responsibility for learning

    14. Students achievement is recognized and rewarded

    15. Students have at least one staff or faculty member that takes a holistic interest in them.

    16. Students have access to equipment and supplies necessary to prepare them for success as defined by the curriculum

(C) Example leading measures

  1. Alignment of board actions to board policy

  2. Policy quality

  3. Quality of curriculum as determined by PDK criteria

  4. Student engagement in instruction aligned to content in an approved course of study

  5. Number of building safety code violations

  6. Percent of program enrollment capacity achieved

  7. Budget within board budget parameters

  8. Percent of positions filled with qualified personnel as determined through an annual audit

  9. Percent of required data in official data systems and properly documented

  10. Students with documented personal and social barriers receiving necessary support services.

Robert Sommers