Suggestions
John Swope
Founder of curricu.me, Digital Learning, Salt of the Earth Podcast
John Swope is an innovator deeply interested in the MOOC (Massive Open Online Course) realm, collaborating with educators and developers to create impactful online courses.
Additionally, his fascination lies in rugged small businesses, devoid of pretense, leading him to co-host the podcast Salt of the Earth with Craig Cannon to engage with and inspire like-minded individuals.
His academic background includes a BA in Business Management & Marketing from New York University.
Professionally, John has served as an Educational Consultant at curricu.me, Co-Founder and Co-host at Salt of the Earth Podcast, Business Analyst at Capgemini, Marketing Director at JumpPost.com, Licensed Sales Agent at Bond New York, Marketing Intern at NextStep Directory, and Founder at Swope Landscaping.
Highlights
Anyone else play Connections? If so, you are almost definitely better than ChatGPT.
In this study, LLMs performed pretty disastrously on the NYT word-game Connections, with an average of 5% win rate for the best LLMs versus 32% for expert humans and 18% for novice humans.
It’s a game that at first blush you might think ChatGPT would be good at, since it involves words and associations. But the game intentionally promotes orthogonal or unusual thinking. Usually, the first pattern you recognize is not the right one. This is challenging for an LLM that likes to find the most predictable answers.
Another reason not mentioned might be AI’s difficulty with “Abductive Thinking”. That is, making a reasonable guess and then checking it, and revising if your initial guess is incorrect. Humans are quite good at this, but it is quite challenging for an LLM that learns patterns from huge data sets and then produces outputs in a most-probable-next-word format.
Will be interested to see this study redone when ChatGPT-5 and similar are released.
https://lnkd.in/ezyRFmhy
We don't have many absolutes yet when it comes to Assessment using GenAI. Some examples from some data I'm collecting on a project.
For context, the project is testing how well an AI can provide rubric-guided feedback and rubric-guided scores on a student-provided report when the AI gets a sample answer and a grading rubric.
Absolute: "AI is expensive" Reality: The cheapest run in our data is 6/10 of one penny. The most expensive is $0.61. That's a 100x difference. The results are different, surely, but one is not 100x more useful than the other.
Absolute: "AI is inaccurate" Reality: It might be. The most "accurate" AI model was an average of <2 points away from human scoring. The least accurate was 38 points away on average. They might both be inaccurate, it depends on whether the feedback they provide with those scores make any sense. We haven't tested that yet :)
Absolute: "We'll find the best model." Reality: The models here don't really matter. Each model will behave differently depending on the use case. Each will have positives and negatives. The reality is we've got to be systematic about finding the right one for our use case. And hopefully remain flexible to be able to adapt as our use cases and the AI landscape changes.

