The Refurbishing of Artificial Intelligence

Hello everyone, and welcome to 2024!  Hope you had some happy and restful time with family and friends over the winter break.

Back in December of 2022, I published a blog entry titled "How AI Will Save Education."  It was just a few weeks after ChatGPT 3.5 had debuted on November 30, and as I put it back then, it "exploded into our zeitgeist seemingly out of nowhere, and it feels like something foundational has rapidly and irreversibly shifted in education...indeed, the world."  There was a lot of strong feelings in the air: wonderment to be sure, but also fear.  And there were many questions.  What does it mean to be a "writer" or an "artist" when AI seemingly has the capacity to be creative?  How does teaching look differently in such a world?   Perhaps most worrisome for educators: What can or should I do if students "cheat" with AI?  Is it cheating to use AI?  Will my time be consumed with a neverending quest to catch students passing off AI-generated work as their own?

Certainly, we still lack definitive answers on those questions, nor is it possible to be definitive when discussing something as rapidly evolving as artificial intelligence.  On such a topic, pressing "publish" almost instantly puts you out of date.  But a year later, some answers have emerged. While I won't dare presume to be the oracle or the burning bush of artificial intelligence, I thought it might be helpful to offer my perspective on some AI trends that have caught my personal eye, and share some selected articles and tools that have emerged in the last twelve months.

The Reality of Students "Cheating"

As it turns out, perhaps we can dial back from the feeling of panic that befell us in December 2022.  As Stanford researchers discovered, "Cheating Fears Over Chatbots Were Overblown" (The New York Times, 12/13/23).   In their recent survey of over 40 high schools in the United States, the amount of students who admitted to have cheated at some point has remained the same 60-70% range that it has been for years.  More fascinating, "[m]any teens know little about ChatGPT...[a]nd most say they have never used it for schoolwork."  Two sets of numbers from the research jumped out at me.  The first is that when looking at all U.S. teens, 44% said they only knew a "little" about ChatGPT and 32% said "nothing at all."  That's three out of four students!  The second shows what could become a disquieting trend of racial and socioeconomic inequity.  When deconstructing the total teen numbers into White, Black, and Latino demographics, the total numbers of "little/nothing at all" are relatively close.  However, when broken down by group, you can see the awareness of ChatGPT inching upward for Whites (50% know at least a little about the tool, versus 27% knowing nothing at all) when compared to Blacks (35%/44%) and Latinos (42%/37%).   Household income makes the disparity even stronger. For households under $30,000, students that know "a lot/a little/nothing at all" about ChatGPT is 11%/30%/59%, compared to the students from households $75,000 and over:  26%/50%/24%.

Does this reflect that ChatGPT is being blocked more at schools in "certain areas"?  Are educators trusting, and teaching AI to, some students more than others?

In a separate set of experiments in September 2023, Stanford researchers explored how well AI cheat detectors were working.   The short answer: not well.  There were a number of both false positives and false negatives.  In fact, when the scientist simply asked the chatbot's generated essay that had been previously detected as AI to "elevate the provided text by employing literary language...[d]etection rates plummeted to near zero."  Another disturbing finding: actual essays written by multi-language learners were often falsely flagged as AI.  This is likely because the more simplistic, emerging language skill of such students seem to reveal the bias of the AI detection programmers: simple text "must" mean a machine wrote it.

Of course, this issue of cheating doesn't address the powerful opportunities to ask a chatbot AI to generate an initial draft of a paper for the student to improve and revise, or to have it provide major points to consider in an argument, or the need to make sure AI is properly cited when used.  Passing off someone's work has always been cheating, whether you Google an essay and pretend it's your own, or when a machine (or human!) writes something for you and you just put your name at the top.   What is important is for students to learn, debate and practice the ethical use of AI.

So, in summary: students may not be using AI chatbots as much as we think, and student cheating with AI (and the software that detects it) is a complicated topic at best.  In fact, we may need to think more closely on how AI is being permitted on school devices and networks, if enough awareness and ethics of AI are being taught, and at the most basic level, reflect on how our concern of "cheating" may be unfairly manifesting itself into inequitable practices in our classrooms.

Proactive and Positive Uses of AI in Teaching and Learning

On a more positive note, there have certainly been numerous reports in the news and research of the potential of AI having a positive impact on learning...or at least approaching it with cautious optimism. Here are a few recent examples: 

• Carnegie Learning has launched several AI tools to help students and teachers. In particular, LiveHint AI is providing real-time math tutoring support.  The tool helps coach students to understand the problem-solving issues of a difficult problem, not simply giving them the right answer: “ 'When students use it, they have the ability to provide line-by-line feedback. If there’s any particular response that they either like or don’t like, [the students] can comment on it,' Carnegie Learning Chief Data Scientist Steve Ritter said. 'And then overall, in terms of the overall quality of the session, they can also comment on that.' "  According to Ritter, the tool has been received positively by faculty and students. ("This Pittsburgh edtech company uses AI to help kids learn math amid ‘uncertainty’ about the tech," technical.ly, 1/4/24)
• Harvard researchers have been studying the impact of AI-generated feedback on students.  The results were positive for teachers, but mixed for students.  Teachers appreciated how AI sped up their ability to respond, as well as gave them ways to customize their feedback to a student's needs.  Many students appreciated how such feedback improved the feeling of a perceived "caring classroom culture" (likely because it was timely and seemed to be responsive), but some struggling students felt the feedback was "unhelpfully short and insensitive."   ("Harvard researchers explore how to use generative AI for student feedback," WBUR Public Radio, 12/27/23)
• The University of Kentucky's Center for the Enhancement of Teaching and Learning has published some AI resources on its website.  In particular, I'm impressed with their "Course Policy Examples," which give an instructor options and exemplars for clarifying whether AI is permissible in their class, from "No [Student] Use" at all, to "Conditional," to "Unrestricted."  

From Prompt Professors to Pragmatic AI Professionals

When the generative chatbots came out in full force in early 2023 (ChatGPT and its rivals like Google's Bard, as well as various graphic-generating AI), there seemed to be a brief time when the educator who was an expert in "prompt engineering" might be the next new thing.  Certainly, we soon became awash in websites and PDFs offering effective prompts to copy and paste for our own AI adventures (such as this one).  What it reminded me of was the pioneering past of other Internet tools.   Do you remember the halcyon days of Geocities, when the idea of publishing your own personal website made us hungry to HTML code?  Or the beginning of Google Search, when we all were ablaze with Boolean operators and the power of a well-placed parenthesis or quote mark?  This was grand stuff for early tech adopters, but for most of us, the effort of process overwhelmed the product.  We don't want be an expert HTML coder unless we plan to create sophisticated websites full-time; we just need a decent templated site that gets our message across.  We prefer a natural language inquiry that gets us the right result, rather than pondering whether it is better to type "AND" instead of "OR" in a search engine.  In short, it is the content that is king. 

In the same way, the appetite for using an AI tool like ChatGPT is also shifting.   We appreciate the power of AI, but want to concentrate on where the vehicle takes us, not so much the vehicle itself.   What that has led us (in a remarkably short amount of time!) is into a world of tools that refurbishes artificial intelligence into an interface that is more inviting and user-friendly.   The word "refurbish" may at first bring to mind a negative connotation, if you only think of the broken device that is fixed and resold at an electronics store.  However, I'm leaning into the original Oxford Dictionary definition of refurbish as renovate and redecorate.   That is an almost literal description of what many new AI tools are doing.  They often take the same "raw" AI code -- such as ChatGPT -- and put it in a renovated, redecorated package that is easier and often more effective to use.  In fact, if the tool has a premium tier, people are now often willing to pay for such convenience, regardless if or whether the raw AI engine underneath the tool's hood is free or relatively inexpensive.  There's a bit of irony at work here; many of the same people (especially educators) struggling with their "buy in" for AI in 2022 are now buying AI.

Here are just a few AI tools that offer outcomes that would have seemed impossible for technology to achieve just a few years ago.  (While there are not necessarily education-specific, it takes the smallest leap of imagination to see how they could be used creatively for learning by a teacher or a student in the classroom!)

• Beautiful.ai.  An AI-assisted presentation tool, this is particularly powerful in creating infographics within seconds.
• Synthesia.  This can not only create videos with AI, it can insert a virtual lifelike avatar that speaks the script you type for it, in over a hundred languages.
• Owlift (formerly called "Explain Like I'm Five").  The tool takes what could be complicated subjects and adapts the explanation of such topics for an audience of varying needs.  Note that you can not only toggle how you're feeling on the topic ("pretty dumb/dumb/smart/pretty smart"), but also whether you want the reply to be sarcastic or not.  The snarky language aside (which arguably may not be appropriate for a young student to interact with), the nuanced functionality of the tool makes it worth mentioning.
• I also need to mention that many popular online tools now include some sort of embedded AI tool, from Zoom to Canva to Google Docs to the writing of LinkedIn posts.

From the website Owlift (formerly known as Explain Like I'm Five).

This refurbishment concept can particularly be seen in educational tools.  It is no small thing to save me time from constantly having to tell AI at every new prompt, "I'm an educator," so that it shapes its answer accordingly.  As such refurbished and personalized AI becomes increasingly sophisticated in the near future -- it will soon know and remember that I am a fourth grade teacher who needs a science lesson every Tuesday, and which of my students will need a translation in Spanish -- the possibilities for AI to become my "artificial instructional aidebot" loom larger and larger.

Here are some promising education-specific tools, with some accompanying cautionary tales:

• Toddle.  A learning management system I first discovered at the Aurora Institute 2023 conference, it was the first LMS I've seen that incorporated AI into its platform.  When the LMS has been preloaded with your district/school curriculum, it can leverage this (along with information about your grade level, content taught, etc.) to have very specific results within Toddle itself, which can then be easily exported or incorporated into your classes.  (And Toddle is not unique in its AI upgrade; since I first encountered it, I've seen several more learning platforms add a version of AI to their toolbox.)  It's great to keep your AI generation within the same browser tab as your LMS!  It should be noted, however, that the purchase of Toddle does not automatically include AI, as it is a premium feature (and likely this would be true of other LMS's that add AI as a feature).
• Curipod.  Another AI-powered presentation tool, but more focused on student and teacher usage.   By entering in your prompts, a lesson plan in the form of a multi-slide deck with built-in interactives for students is generated.  The features and the name remind me of Nearpod, and like that platform, a student can join a teacher's live session with a PIN.  There is a limited free account available.
• LitLab.  A tool for creating "AI decodable books" to help primarily with early literacy.  Once perimeters are given, an online book is generated with customizable text and images, which can then be shared with students via its URL.  However, the images, while clearly AI generated, are not very sophisticated, nor are they specifically tied to what is occurring in the text of its particular page.  While currently free (if you nab one of the limited initial seats available), the platform will become a paid service in June 2024.
• Magic School AI is impressive by having a one stop shop approach to AI integration.  You can discuss ideas with Raina the Chatbot, which can then export your result as needed.  Additionally, there are several dozen AI "Magic Tools" that lean into a specific need.  Examples include "Text Leveler," "YouTube Video Questions" (give the video URL, and you can create a custom assessment based on the video's content customized by grade level and number of questions), "Choice Board (UDL)," "Project Based Learning" (create a plan), "Rubric Generator," and much more.  (Again, it should be pointed out that free chatbots can generate many of these outputs, but perhaps without the specificity and convenience this platform can achieve.) While Magic School offers an initial free trial, it does eventually cost to use.

Magic School AI

One of Magic School's "Magic Tools" is an "IEP Generator."  This brings up one of the biggest takeaways and pieces of advice about using AI, regardless of the platform or type: be mindful of what you put into a prompt, and be careful how reverently you treat its output.  To further explore this example, I asked my OVEC colleague and special education teacher Dr. Debbie Mays for her opinion, and Dr. Mays shared some thoughts she and her ECE colleagues have had on using AI for creating an IEP: "We have the concern of confidentiality and information being exposed [when personally identifiable information is inputted into an AI tool]. Also, if AI just considers federal requirements, it may miss the unique state regulations and statutes that do need to be considered and followed. It has to do both, as well as follow district policies and procedures. Finally, the intricacies of an IEP – all the connections and 'threads' that need to be addressed for that specific student throughout the program -- will probably be missed, opening the district to legal ramifications." Ultimately, Dr. Mays gave her personal "thumbs down" on AI-generated IEPs, but acknowledged "there may be ways to use it. The trick is a teacher needs to know how to write an appropriate Individual Education Program. If they don’t know how and need help, they may not know how to fix one either, if AI started one for them."  Again, while applying AI to IEP creation is a very specific use case example, it illustrates the general need for educators to practice discernment.   AI is not infallible, nor is it always "right."

As I wrap up this entry, we can see that the last year has proven the reality of AI is somewhere between an apocalyptic destroyer and consequence-free manna.   In May 2023, the U.S. Office of Educational Technology published "Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations." In a brief handout with the paper's core messages, a comparison was made between electronic bikes and robot vacuums.  Robot vacuums may be treated as somewhat independent machines, with a "mind" of their own; such machines may be disruptive in the way that they can substitute for a human completing the task it was designed for, or perhaps more ominously, replace the human worker that was previously needed for such janitorial labor.  However, the OET would assert that it is more accurate to think of artificial intelligence as not our eventual robot overlords, but as an electric bike: "We envision a technologically-enhanced future more like an electric bike and less like robot vacuums. On an electric bike, the human is fully aware and fully in control, but the burden is less, and their effort is multiplied by a complimentary technological enhancement."

From the OET's summary handout of their May 2023 full report.

In the world of our educational future, we should hope to use more artificial intelligence "electronic bikes" for ourselves and for our students.  Let's embrace AI to augment and enhance what makes us human so we can dare to dream, think, make, and create in even more innovative ways, and by doing so, transform the learning experience.

Update 6/15/24:  The website "Explain Like I'm Five" is now Owlift.  I edited the entry in the appropriate spots above.

 

How SBG Led Us to Empower: The Power of Evidence (Part 2 of 2)

This is part two of a two-part blog series.  When our district began a move to standards based grading (SBG), we realized that we needed a different digital tool.  In the previous blog entry, I discussed how the limitations of a traditional grading system led us to SBG, and ultimately, Empower Learning.  In this entry, I give an overview on the grading features of Empower and how it integrates with our standards based learning philosophy.

While Empower is a fully functioning learning management system, its core capability of capturing student evidence and calculating this for overall standard scores is really the "crown jewel" of the tool.

But first, what do you see when you log into Empower?  There are three main interface tabs.  The Instruction Tab is where you can see classes (either fed from your student information system or custom created groups), as well as create and assign instructional materials like Activities and Quizzes.  The Reporting Tab is where you can output progress reports and export other data.  The Scoring Tab is where "gradebooks" are located, and the place where evidence and standards are scored.  (Remember, as is necessary in SBG, these scores are based on established Mastery Scales -- for Shelby, these range in whole numbers from 0 to 4.)

Example of a test class/group (with a single student) and a sample gradebook with one standard shown.

Empower has two types of gradebooks.  The first is a class's default gradebook (also called the "master course playlist" by Empower), designed and managed at the district level, that designates the prioritized standards for the course.  This master gradebook determines many important reporting and analytical aspects of Empower, such as progress reporting and the final course grade calculation.  (In a knowing nod to a traditional system needing a final "number" grade at the end of the year for secondary classes, Empower can take the end of year scores of all the prioritized standards and average them together.)   The second type are gradebooks custom created by the user, in order to help teachers focus on a smaller lens of standards (perhaps just a few standards addressed in your Unit One) or to make cross-content project assessment much easier (think of PBL possibilities!).

Under the Scoring Tab, you can toggle between "Score Standards" and "Score Evidences," which show the interrelationship between the two -- they are truly two sides of the same coin.  Under "Score Evidences," you can see tasks that have been targeted to various standards and score these individual evidences accordingly.  These may have been created in the Instructional area, such as Activities and Quizzes, but one of my favorite Empower features is the ability to make "Quick Evidence" -- within a button push and a few clicks, you can quickly create and score evidence. (Many learning management systems only allow grading of digital activities created inside its system, but Empower allows you to score and assess virtually anything in seconds -- analog activities, observation of classroom discussions, and so on.)   Under "Score Standards," a teacher can score an overall standard if they wish.   However, one of Empower's most powerful tools is the Marzano True Score Estimator (MTSE), which is accessible within a gradebook's "Standards" toggle side.  You can use it to look behind a particular standard, and not only examine the body of evidence each student has demonstrated, but see how Empower calculates and projects an overall standard score based on the evidence score numbers and when the evidence occurred.  To do this, Empower actually runs the evidence scores through three different formulas -- average, Linear, and Power Law -- then recommends one or more as the most reliable for the given set.  This creates a completely different nuance to grading (which constantly updates as new evidence is entered), and a very necessary one when deciding and defending a student's mastery of a standard.

Let's briefly walk through these formulas.  We are very used to averaging, but I will highlight here that it is the only one of the formulas that is indifferent to when the evidence scores happened -- in other words, whether the scores were from two years ago, last September, or yesterday.    Linear is looking for a trend line; if we graphed these scores, are we trending upwards or downwards?  Therefore, more recent evidence is important when determining the direction of the trend.  Lastly, Power Law is based on an intuitively typical human learner.  When most of us encounter a new concept, we struggle to understand it, and if we were "scored" on our skill or knowledge, would likely score low with 1's and 2's.  As time and experience went on, we would get better, likely scoring 2's and 3's, and in time would achieve mastery with 4's.   Power Law takes that progression into account by mathematically counting earliest scores with considerable less "weight," and mathematically counting most recent scores with considerably more weight.  In short: for Power Law, how you just performed is much more important than your potential low scores in the early part of your learning.

Take a moment to compare these multiple, nuanced levels of assessing learning to Part One's story of Timmy and his traditional grading conundrum, who like many students struggled with such a low test score early in a unit that it didn't matter that he aced the final test; in a traditional system of averages and percentages, only a student who is freakishly perfect from the start would maintain a high grade, and a student who starts off poorly cannot avoid the demotivating academic hole they can't statistically crawl out of.   Empower changes the entire approach to looking at what grading is communicating, and at least as importantly, gives students specific guidance on how to improve their learning -- they not only know their overall standard strengths and challenges, but how their evidence behind a standard has an accumulating history, and the importance of today's performance is actually calculated as more important than whatever struggles the student may have had earlier.   (There is an additional positive side effect of this as well: students that perform exceptionally well at the beginning of an academic year cannot rest on their good laurels and coast into a passing grade when "spring senioritis" sets in and their present performance significantly drops.)

I admit that when I first began learning and teaching others about how Empower calculates scores, I struggled to understand and explain this well to others.  As the school year began, I created a Scoring Calculations Doc to help explain how this worked to our district staff.  For the sake of this blog entry, I'm only bringing the Doc up to point out one common concern that parents, students and even some teachers have: in a system of mastery scales and standards based grading, will final course grades, GPA's, etc. go down for most students?  In the Doc, I attack this head on by using a fictional set of four students and examining their evidence scores under a single hypothetical standard.  Luke is the Marzano ideal student of human learning, who starts with scores of 1's, then 2's and 3's, and finally 4's.  Han is the opposite: he started at all 4's and wound down to 1's in several recent scores.  Lando is all over the place scorewise for a while, but settles out and eventually scores consistent 4's by the end of his scoring timeline.  Lastly, Leia has mid-level scores fairly consistently, although she tends to score 3's (including recently). 

When comparing the simple average of the scores versus their Linear/Power Law projected calculations, the above table shows how the idea of averaging conflates the academic "stories" of these four students as equal, when we can see this is limiting at best and strongly inaccurate at worst.  Interestingly, in a mastery SBG system that goes beyond merely averaging, three of these students do as good or better than they would in a traditional grading system.  With SBG, there is not only more of a chance for a student to score a higher GPA, but such a GPA may actually reflect a true accomplishment of (mastery) learning instead of just an accumulation of points.  Out of our four fictional friends, the only student that does worse in SBG -- Han -- is the one student we can clearly see is "bottoming out" or having a bad case of the senioritis we mentioned earlier, but instead of facing an academic penalty for this in traditional grading, Han would unfairly coast into the same grade as his peers.

Empower is an example of a powerful digital tool that is a potential game changer for assessing student learning.  Of course, a learning management system is merely a holder of digital data entered by humans -- it cannot replace a teacher's professional judgment and relationship building necessary for student learning and growth to occur.   Empower can calculate scores quantitatively, but has no ability to determine the qualitative nature of the assessments themselves; this is where "the art of teaching" and a teacher's discernment come into play.  (This is one reason why Empower always allows a teacher to determine an overall standard score, even if that means overriding what Empower suggests based on evidence scores.) Empower does not magically make teachers effective in standards based grading practices, or make their assessments automatically any deeper, more rigorous, or authentic.  Excellence in SBG requires dedication, professional learning, and time.   Empower does not short cut this or offer a silver edtech bullet.  However, I do have to say that in the six years I have been a part of education technology in Shelby County, there have been few if any digital tools that have pushed and challenged our teachers to teach transformatively as much as Empower.  It is impossible to use it fully and faithfully and still maintain a traditional grading mindset.   In that same vein, as we continue to support and fulfill our current Strategic Leadership Plan, tools like Empower are a valuable asset in order to move forward in our goal to enrich and enlarge our capacity to personalize our students' learning with a "mastery of standards" philosophy, and leave behind the idea of school as a game of compliance and points.

How SBG Led Us to Empower: The Tyranny of 82% (Part 1 of 2)

Happy holidays and welcome to 2020!  Hope you enjoyed some well deserved time off during winter break.  I have often talked in Edtech Elixirs how digital tools should help us serve an academic objective.  For our district, the need for effective standards based grading (SBG) began a search for a transformative digital tool.  In this blog entry -- the first of a two part series -- I discuss how the limitations of a traditional grading system led us to SBG, and ultimately, Empower Learning.

I'd like to start this entry with what I like to call "the tyranny of 82%."  While it's fictionalized, it's based on the reality I have faced in the past as a classroom teacher in a traditional grading system.

A parent walks into a classroom to discuss her son's current grade with his English teacher.  "I'm worried," the parent frets.  "Timmy currently has a 82%.  That's a B.  A low B.  What can he do to raise it to an A?"

"I'd be happy to help," the teacher smiles.   "I have some practical advice for him.  He should work 8% harder."

"But...how?" the parent asks.

"Well, let's say right now he studies 50 minutes a week.  Timmy should start studying 54 minutes a week. That's eight percent more effort."

"Oh yes.  I get it now.  Anything else?"

"Yes.  He should always get a high grade on every assignment, right from the start.  Let me give you an example.  Last week we started a new unit on a concept he had never done before.  He got a 20 out of 100 points on the first big assessment.  At the end of the unit, he aced a final assessment with 100 out of 100 points.  But it was too little too late.  All that effort at the end and he only got an average of 60 percent in summative assessments.  Of course, if it makes you feel any better, Timmy could have gotten a 100 on the first assessment and 20 on the last one and would have gotten the same grade overall.  He should have scored high all along, you see?  The math of averaging doesn't lie.  And I average all students, all the time.  It's only fair!"

"Ahhh!  Very fair. Thank you for the practical advice!"

Of course, the above is farcical.   But as a classroom teacher analyzing an overall course average in a traditional grading system, I wish I could have given feedback to a student or parent as confidently as the fictional teacher above.  On a good day, I could have pointed to a well-designed rubric for a particular assignment to show how you could improve from good to great, but therein lies the problem -- a rubric helps explain the score on a particular assignment, and not really an overall course grade.  For a typical student that wants to increase their class grade by a level or two, I would usually fluster through a conversation on points -- if only they had gotten a bit higher on this quiz or that project, or would get full points on an assignment next week, their grade would go up.  For failing students, it would be a much easier conversation, since it was often a problem of zeroes, and therefore the talk was about completion and compliance.  If Timmy had just turned in a few more of his homework assignments and that big paper from last week, he might not have a F!

If all this talk of points makes school with traditional grading sound like a game to either win or lose, it often is, and therefore no teacher should be surprised when students try to manipulate such a high-stakes and highly subjective system.   (Don't even get me started on how "extra credit" skews the validity of academic data even further, or how enough zeroes can create a statistical hole that even the most motivated student would not be able to climb out.)  However, the problem in all of this talk of points and completion and compliance is the absence of what should be the true question to consider: how does 82% reflect what Timmy actually knows in English?

The answer: it doesn't.

Percentages aren't always impractical, of course.  If I have a 82% approval rating, there was a poll result that revealed 82 out of 100 people like me.  If a medical procedure has a 82% success rate, I can take comfort in the fact that there are "only" 18 out of 100 case studies when the procedure fails.  But if someone said that I am 82% in my husband skills, or a podiatrist is 82% in his medical knowledge, we would laugh at the absurdity of how non-informative those percentages are for such broad concepts.  Yet no eyelids are blinked when a percentage is applied to an entire course and we say a student is 82% in English, or Geometry, or Physics, or U.S. History.

We should note something about letter grades before we move on.  An A-B-C-D-F system is not in and of itself necessarily bad -- I would rather deal with a five point range system than the hundred point range system of percentages -- but what is troubling is often the vague understanding and frequent inconsistency of what a letter grade means.   For most traditional grading systems, the letter grade is simply the mask that a school or district's percentage wears at the ballroom school dance of public opinion, in an attempt to feign academic consistency and conformity. To take the simplest example of how this mask conceals more than it reveals, consider that even the scale used to translate a percentage into a letter grade can vary district to district in the same state, or even school to school in the same district.   An 80-89.9% could be a B in one location and 86-92.9% could be a B in another. So, again: How does a B reflect what Timmy actually knows in English?

If we take it as a given that a game of percentages is not the best system of grading -- and many of you were likely nodding your head about that way before this point in the blog entry! -- what could replace it?   First, we need to start with the question that Ken O'Connor poses in the short video (2:36) below: "How confident are you that the grades you students receive are consistent, accurate,  meaningful, and supportive of learning?" If we are not confident in what we have, let's find a better one.  To paraphrase from O'Connor's video, if students are "playing the game of school," let us at least make sure it is a learning game and not a grading game.

It was the pursuit of a more effective grading system that led Shelby several years ago to begin a transition to Standards Based Grading (SBG).  While all teachers have theoretically planned instruction around their state standards for many years, SBG looks at measuring student academic progress through the lens of how they are doing in each standard that is pertinent to a particular class. As we explained in a recent handout to parents,  "Course standards should answer the question: What is it we want our students to know and be able to do?" In order to articulate where a student is currently in their standards in as clear and consistent way as possible, we have created mastery scales (whole numbers from zero to 4) to go with these standards.  These scales work macro and micro: not only are they used to assess a specific task or evidence of learning with a score of 0 to 4 in that standard, the same scale applies to the overall standard when assessing the student's body of evidence.  Effective SBG practice is an important step on the journey to mastery learning -- when students can clearly and meaningfully apply their knowledge in new contexts -- and, eventually, a true competency-based education (CBE) system.  Our current Shelby Strategic Leadership Plan 2.0 has a goal of a CBE system by 2022.  (For a good starting place on learning more about competency-based education -- in particular, a new updated definition of CBE -- I highly recommend checking out Aurora Institute's new paper released in November 2019.)

When a teacher does SBG well (after support, experience and practice),  SBG is clearly much better at meeting O'Connor's four characteristics of effective grading.  Let's look back at the 82% conundrum.  If the reporting instead indicated how well Timmy was doing in his English standards (rather than the accumulated or averaged points from a gaggle of assignments), areas of strength and challenge would be much more clear.  We can not only see Timmy's success in standards with a 3 or 4 overall score, but we can quickly focus on Timmy's potential struggles in standards with a 1 or 2 score.  By reviewing mastery scale language, we can determine what it would take to improve in those struggle areas.  So let's use O'Connor's language to critique SBG. The common mastery scales keep us accurate and consistent across teachers and schools; discussing overall standard scores is a much more meaningful way of answering "How is Timmy doing?" than a vague overall course percentage or letter can achieve; student learning is supported when the scores on standards can lead to clear action steps of improvement.

The philosophy of SBG is not the stumbling block for most educators.  When explained like I did above, who would argue that the traditional grading system is more fair than SBG?  The issue is in the application -- how to track and monitor SBG, especially over time.   While teachers have done this without digital help, it is time consuming and difficult.  You can "hack" traditional online student information systems (SIS) to attempt SBG, but such tools are often teacher-centered and optimized for linear assignment record-keeping.   A few years ago in Shelby, our teachers asked for something better -- not only in an online gradebook, but in a learning management system (LMS) that had SBG as its centerpiece.   After a committee of teachers and admin reviewed several platforms, Empower Learning emerged as the most comprehensive digital tool for several reasons:

• It is student-centered.  Not only does it allow for better student advocacy of seeing their academic performance in all classes over years of their academic journey, it allows all of the student's teachers to see all of his/her academic performance in all areas.   (Imagine trying to make an advisory system without such a transparent system of support!)   Compare this to a teacher-centered SIS that is built to record assignment completion and is silo'd to begin and end information for mainly just that teacher, for just that class, for just that school year.
• Personalized learning can be done well and help a teacher use their time more effectively.  Without academic information to keep it rigorous, personalized learning could potentially become all voice and choice without rigor and equitable need.  Personalized learning is best when both the teacher and the student can easily see areas of mastery (if you have mastered all fourth grade standards, why not begin on the fifth grade ones?) as well as standards with low scores requiring intervention.  Personalized learning can also be time consuming to plan and facilitate for a teacher, so digital tools to help streamline this is important.
• Behind an overall standard score, you can see the historical body of evidence that led to that score.  I've seen prior gradebooks where the standard score may have changed from 2 to 4 to 3 over several school weeks, yet it is not clear why or how the score changed -- instead of a real-time story, you can only see what the standard score is right now (or at best, 3 or 4 "snapshots" at the end of quarterly terms).  As a classroom teacher, I patted myself on the back when I made assignments that were standards-referenced (i.e. merely tagged to a standard) but I did a poor job of analyzing how those assignments could lead to a determination of mastery of any particular standard.  In both of these cases, it is extremely difficult to do any better without a digital tool like Empower.
• It is a "one stop shop" of LMS needs.  While we found some tools that were decent digital gradebooks, few were able to offer the ability to do what a typical learning management system like Schoology, Google Classroom, Edmodo, etc. can do, such as creating quizzes, assigning work for student submission, and housing a collection of resources.  Of course, the opposite is also true -- for example, Google Classroom offers no gradebook beyond one with traditional points and percentages. Empower organizes both instruction and scoring under one roof.

For more information about the Scoring part of the Empower LMS, visit here.

As we wrap up this entry, I want to return to Ken O'Conner's video from earlier.  He points out that in a truly effective grading system, a student should not ask "What can I do to improve my grade?" but rather "What can I do to improve my learning?"  In Part Two, my next entry will go a bit deeper into how Empower integrates standards based grading and is an important tool to help us shift our Shelby students from the former to the latter.

Author's note: Shelby County Public Schools has invited Ken O'Connor to come to the district in January 2020 to talk to our teachers and community about grading practices and, in particular, the power of SBG.

Part Two of this series is available here.

BIg Transitions and Changes

It has been four months since my last blog entry!  I've had gaps between entries before, but that's a pretty long one.  I thought it would be a good time to reflect on what has been going on as well as look ahead to the future.

In order to be transparent in my sharing, I need to first flash back to July 2018.

April, my beautiful wife of 16 years (we have actually been together since 1995, if you're keeping score at home), was trying to schedule a procedure when her doctor's office stopped her in her tracks. 

"It says here you are over 40?"

"Barely," she replied.

"Well, you've never had a mammogram."

And I don't want to start now, she thought.  "I don't have the slightest symptom and I'm very healthy. Can't we just put it off for a few months and do this other stuff instead?"

"Nope.  Insurance won't allow you.  Sorry."

With that, April had her very first mammogram -- which discovered an abnormality.  Surprise turned to concern and disbelief as a biopsy revealed Stage 3 breast cancer.  And for April, myself, and our two daughters, the long fight began.

I am pleased to say that we are at a happy crossroads.  My brave, persistent wife -- one of the grittiest people I've ever met -- has recently completed treatment of chemo and radiation, and although we have a few surgeries ahead, we feel the worst is behind us.  It has been an agonizing, exhausting year, and I suppose this is a very long winded way of saying that as the school year wound down, I rationed my energy and I took a step away from things that could go into suspended animation -- like this blog. 

However, if you're not willing to give up on me, I'm not ready to give up on Edtech Elixirs just yet!  I know there will definitely be major reasons to blog in 2019-2020 on new edtech tools and the occasional pedagogical discussion.   To take one example, I'm sure I'll talk a time or two about Empower Learning, a new learning management system we piloted last school year and as of July 1 has replaced Schoology in our district.  However, Empower is much more than just another LMS.  Its ability to reference standards and track mastery learning for students over time gives us a powerful tool to help Shelby County Public Schools reach our competency-based education goal in 2022.

While personal interests of mine have previously crept into this blog, I imagine I will surprise most readers with sharing our family's struggle in such a public way.  For those concerned I will make Edtech Elixirs into an online diary, I promise that I will return fully to the world of edtech in the entries to come.   That said, I do want to end this entry with a sharing of gratitude as well as a plea.   Firstly, there have been dozens of family and friends and co-workers that have given our family such unbelievable support over this past year.  We simply would not have survived without you -- thank you from the bottom of our hearts.  Secondly, as my wife's story illustrates, putting off a mammogram, or any regular cancer screening, can have tragic consequences.  In a real way, we got lucky.  Please make sure you and your loved ones don't rely on luck to keep healthy!

If you are looking for cancer organizations to support:

Norton's Healthcare (serving Louisville and Southern Indiana) has been an incredible healthcare partner and resource in our fight against cancer.  Here's ways you can give time or money.

April is passionate about P.ink, an organization where tattoo artists donate their time and material to breast cancer survivors, particularly in covering up or beautifying mastectomy scars and breasts both unreconstructed and reconstructed.  More about their organization is here.

Student Journals, Personalized Learning Plans and More: The Power of HyperDocs

Happy New Year 2019!

Are you needing students to make a journal, as well as see and possibly comment on each other’s blog?  Finding a tool with easy visibility and simple user interface inside of some kind of protected space can be difficult to find, in addition to doing all of it for free.  (While Blogger is an excellent free Google tool -- obviously I'm a fan! -- I know it is blocked by some districts due to monitoring and privacy concerns, or considered to be too complicated for their younger students.)

 

One way to have safe, free, and easy blogging, especially with younger students, would be to utilize and organize Google Docs in some kind of “HyperDoc” stylized fashion.*  How much you do, versus how much the students do, will determine the amount of heartache and labor involved to set it up.   The steps below assume a higher capability/age of student, perhaps upper elementary or older (which will make the least amount of work on you); adjust and scaffold the work flow for K-2 students as needed.

 

1.  Create a Google Doc that serves as a journaling “template.”  Be sure to make a space/field at top for them to type their name and a way to date each of their entries. Make it a sharable URL as "view only."  Copy the URL.
2. Do the “force copy trick” with the URL so that, when the URL is clicked, it forces the user to copy to their own Drive.
3. Provide this “force copy URL” inside of your learning management system (LMS). As students click the link and save the copy to their Drive, make sure they change the words “Copy of” inside the title of the Doc to “First and Last Name” (for example, “Copy of Journal Log” becomes “ADAM WATSON Journal Log”).
4. Have each student open the sharing link on their new Doc.  Make sure the permission is “[District Domain] Users Only, view and comment.”  The student should copy this URL.
5. Create a simple Google Form for students to submit responses; again, link the URL in a LMS.  Ask for their names, but make sure the Form automatically collects their email addresses.   The students should paste their journal’s sharable URL as an answer to one of the Form’s questions.  (Note: organizationally you may want to create a different Form for each class; the reasons why will become obvious in the next steps.)
6. Within the responses tab of your Form, make sure you create a Sheet to go with the Form’s responses.  You now have ONE spreadsheet that has links to ALL of the journals of the students.   Now, make the Sheet “view only” and provide that link inside a LMS for your students to access.
7. By clicking on the Sheet’s URL, you now have an easy way for students to access each other’s Journals and read them (without the students, or you, creating hundreds of sharing permissions).   Additionally, if they Comment, they will be identified (no anonymous comments allowed) and some privacy is maintained as students have it restricted to visibility only within your domain's users.   By having “view and comment” instead of edit rights will also prevent students from intentionally or unintentionally writing over each other or even nuking each other's journal with deletions.   Of course, multiple Sheets also make it easier for you to look at the various journals class by class.  Bonus #1:  collecting the journal URLs in this way avoids receiving all of those inbox-cluttering sharing email notifications.  Bonus #2: the Form and Sheet  automatically organizes the journal links without you spending time creating multiple Google Folders and organizing all of the student journal Docs inside them.
8. One last step/tip: if students are to respond to each other's journals, make sure students understand not only what a valuable “uptake” Comment is (beyond “that is great!”), but also make the expectations clear.  Should they Comment on someone else’s journal twice a week?  Two different journals, or two Comments on the same journal ok?  Should they eventually Comment on all student journals at least once? Do they need to reply to the Comments made by other students?  Will they be assessed qualitatively (do you have a rubric?), quantitatively, or both?

 

Note that using this same workflow could be applied in other ways you want students to make a copy of a template, share it with you to review, and possibly have others give feedback.  One example could be a Personalized Learning Plan (PLP) that students initiate, periodically reflect and edit, and share with you (and their advisors).  And speaking of personalized learning, if you want to really create an intricate, HyperDoc work flow that could track mastery per standard and per student as well as link to archival evidence, I recommend taking a look at my “SCANMoST” directions.   While SCANMoST was originally created to optimize the 2017 integration of Google Drive into Schoology, it could easily be adapted for Google Classroom or any LMS by utilizing the same “force copy”/template work flow. 

* "Stylized" is an important qualifier here.  I should definitely point out that HyperDocs are a larger construction than merely a heavily hyperlinked online document; as this article by Jennifer Gonzalez defines very well, a HyperDoc is a "digital document . . . where all components of a learning cycle have been pulled together into one central hub."  The potential impact on pedagogy and personalized learning go way beyond just providing a helpful organization structure for a teacher.  Nevertheless, the further you read into my entry, the closer you get to the student-centered "hubs" that would meet Gonzalez's definition of a HyperDoc.

Improved Google Integration with Schoology!

Back in 2015, our district purchased the Enterprise edition of Schoology.  It has grown to be a popular tool (as of a few months ago, our data analysis determined that three-fourths of our K-12 students and teachers are using it).  However, it is not the only learning management system in Shelby.  Google Classroom is popular among some of our instructors, who either use it instead of Schoology, or right alongside it (and Google Drive, of course).  The preference for Classroom usually is for one or both of the following reasons:

• The Google Classroom interface is simpler, especially when dealing with Google files.
• Google Classroom can "pass out" unique digital copies to students as well as organizing the editing rights (incorporating the name of the student in the name of the file, automatically creating folder structures for the files to be housed, and automatically changes the student's editing rights to "view only" once something is submitted).

The first reason comes down to what you prioritize: Classroom is one of the simplest LMS's to use, but at the sacrifice of the depth of options and tools that Schoology can do (especially the Enterprise edition). As for reason #2, that's about to be challenged!  While it has always been possible to use or incorporate Google files in Schoology Assignments, they were not as easy or seamless to distribute as the Google Classroom experience.  However, starting at the beginning of the 2017-2018 school year, this will change.  (It is worth noting that this update affects all users, both Basic [free] and Enterprise.)  I can say without hyperbole it's the most important update I've seen for Schoology since our district began using it two years ago, and I have a feeling it may convert some of our Google Classroom teachers to switch completely over to Schoology.

So, what might you expect this fall?  I participated in an informational webinar today, and here are some highlights.  Before we review the update, it's important to note that students AND teachers need to add the Google Drive app to Schoology if they haven't already (under Resources>Apps), and the user's school G Suite account is the credentials to use.


This graphic gives a good overview of what will change with the new integration:

From the 4/13/17 Schoology webinar.

Now when you create/edit an Assignment, you will see in the box an option to “Assign from App” (Google Drive).   Next, choose the file. You can now search or browse your Google files (previously you could only browse your files).  This creates a unique copy of the file for each individual student, just like Google Classroom does.   Also, just like Google Classroom, the teacher is the "owner" of the file, and a folder is created on both sides (student and teacher) where the new files will live.  Up to the due date, student can edit these files; after the due date, they will only be able to view unless the instructor chooses to unsubmit the assignment.

Only parents, advisors and the student can see the submission (as before with any Schoology Assignments).

From the instructor view, you can then see and easily toggle between different student submissions. 

Instructor view. From 4/13/17 Schoology webinar.

All file types can be assigned in this way (Slides, Sheets, etc.), and their appropriate add-ons, etc. will work in the new Google viewer environment.

In this new viewer, the instructor will see the Google Docs in real time, just as you would if you were looking at the file in a separate browser tab outside of Schoology.

Students can “unsubmit” before the due date if they like.  But once the doc is submitted and the due date occurs, student will no longer be able to edit!  (Again, say it with me: just like Google Classroom.)

In this new Google viewer, "standard" Schoology tools like audio/video feedback won’t work.

Within the same browser window, an instructor can grade with a rubric (using the same Schoology rubric tools as before).  Graded rubrics will connect to the student work.

Instructor view. From 4/13/17 Schoology webinar.

Here is Schoology's one minute "teaser trailer" of the upcoming Google integration:

The webinar that discusses the integration is archived here (approximately 22 minutes).

How will this make life better for teachers and students?   Here are some quick thoughts:

• Using Google files as templates that students complete and customize can be done in seconds. By making the form easy to share and manipulate, students can concentrate on the crucial work of demonstrating their understanding.
• There's really no equivalent tools in Schoology to make content like Docs, Slides, Sheets and Drawings.  Google is excellent at this, so it's a perfect partnership. By keeping Google work inside of Schoology, you can expedite creativity since it will remain "in house" (same tab, same platform).
• No lost time hunting down generically named Google document titles.  Since you name the template, combined with the automatic insertion of the student name in the title, files will be unique and easy to find (and that's setting aside how they are saved in certain folders!).
• For a typical Schoology Assignment, a student is permitted to turn in a submission late, although it will be flagged as such.  However, under this new option, these Google files turn to "view only" after the due date. It is the first time that due dates prevent late submissions in Schoology! Some teachers may use this to improve student behavior for adhering to deadlines.
• Google files submitted via Schoology Assignments means that students don't have to waste time sharing the files directly with the teacher -- and teachers won't have their email inboxes and "Shared With Me" section of their Drive cluttered and bombarded with requests.
• In short, easier and faster work flow means students can concentrate on the thinking and creating, and teachers on assessing and giving feedback.

I can't wait until the fall!  Let me know your thoughts in the Comments below.

Update 4/18/17: I now have added the link to the webinar.

Update 7/27/17: It's here!  Below is a video I made where I demo how it works. I also provided a link to the instructions on Schoology's help center.   Note that you must install the Google Drive Assignments app in Schoology before this will work.  (Shelby staff, I have already installed this for our domain; any teacher with admin rights to a Course can use this.)

Google Drive Assignments app instructions (Schoology Help Center)

Important Privacy Setting: Disabling Roster View in Schoology

Tomorrow morning, I fly to Orlando for iNACOL 2015 as part of a team of Shelby County & KDE presenters about our district's technology integration plan.  Tweets and blogging will surely follow.

Meanwhile, Shelby County is finally ready to roll out the enterprise version of Schoology.  I am definitely becoming a fan of the LMS and eager to see how teachers use it to facilitate learning for their students.  It's a critical piece not only for digital conversion, but to enable other pedagogy to succeed, such as PBL and personalized learning.

However, we have just discovered a critical privacy setting that needs to be changed.  Otherwise, students can see teacher rosters for each of their classes, including profile information on each student.

The steps are easy.  Teachers, please do this as soon as possible.  (Thanks to Gretchen Schell at Schoology for the directions and screenshots!)  Check to make sure you do this for each course you have attached to your account.

Step 1: Select the drop-down of Course Options (see screenshot below) and select “Edit Privacy/Course Settings”

Step 2: In the Permissions area select “No One” for Roster. This will only allow the Teacher (Course Admin) to view the members enrolled in a course. You must SAVE CHANGES to enable this setting.

On the far right side of "Privacy," in the "Roster" row, checkmark "No One."

If you have any questions, please reach out to your school's Instructional Coach or Digital Learning Coach.