I first discovered the Goldilocks Rule in the book Atomic Habits by James Clear where he says:

“The Goldilocks Rule states that humans experience peak motivation when working on tasks that are right on the edge of their current abilities. Not too hard. Not too easy. Just right.”

James Clear

When you take on projects or learn new concepts that are ‘just right’ for your current abilities, you can more easily enter the ‘flow state’ while working where you’re so focussed on what you’re working on that you lose track of time and don’t get distracted by any of the usual things that pull at your attention (ahem, Instagram).

Similarly, Scott Young proposes the 85% rule – a way of determining how you should adjust your learning based on your success rate. He says that, on the one hand, if you get every exercise or task correct while learning, you’ll think you don’t need to make any improvements and you’ll probably overestimate your skill level in the topic. On the other hand, if you’re regularly getting exercises or tasks wrong then you won’t learn enough to be able to improve and your motivation will fizzle out.

The 85% rule is very useful for tasks that have an objective measure of success – such as math problems or coding exercises. In those cases it is clear when an answer is right or wrong and you can ensure you take on exercises where you get 85% of the problems or exercises correct.

How to Apply the Goldilocks Rule to Data Science Learning

Data science is a huge field and there is a lot to learn, with new developments being made and published daily. While on your data science learning journey you may find many new courses and tutorials on topics you know and many that you’ve never encountered before. It can feel much more comfortable to stick with beginner content until you feel ‘ready’ to take on more challenging concepts. However, if you do what’s easy or what’s comfortable, you won’t be able to improve and figure out where your weak areas are.

I encounter this a lot when taking a new course or starting a new book. Many courses and books will often try to cover a broad range of topics that go from beginner to intermediate and sometimes even to an advanced level. The mistake I make is that I think a course must be taken from lesson 1 or books must be started from chapter 1. This is definitely not the best approach. It is difficult to maintain motivation when everything feels so easy. You’re not being challenged or pushed to improve yourself and so you will most likely not follow through with the book or course.

On the other hand, if you take on an advanced NLP course when you haven’t even covered some of the basics of text analysis or neural networks then you’ll probably feel very demotivated by how difficult the course is and quit.

The better approach is to identify what your current skill level is and take on new concepts, exercises or projects that are just at the boundary of your capabilities. Not too hard. Not too easy. Just right. Resist the urge to remain in your comfort zone where things are easy or that you know very well, or to take on things that are too challenging and you have no foundations around the topics that are covered.

How to Use the Goldilocks Rule to Stay Motivated

Something I am always trying to figure out is how I can stay motivated to do things over a long period of time. Even though I love data science and I love learning new things, I still feel my motivation levels running low at times.

While I have not found something that instantly makes me motivated all the time, you can use the Goldilocks Rule as one tool in your toolkit to help you maintain motivation over long periods of time.

The reason that this works is because as human beings we are hard-wired to avoid pain (i.e. very challenging cognitive tasks in this case) and we absolutely hate being bored (which is why so many of us are addicted to social media – dopamine, please).

By working on tasks that are at the edge of our capabilities, we get constant feedback on how we are doing. This means we get a dopamine boost for the things that we can actually get right but are still challenged enough so that we don’t get bored or frustrated.

However, an important part of the Goldilocks Rule is keeping track of how you are doing and measuring you progress. It’s easy to subjectively look at everything you’ve learnt over the course of a day/week/month and say “everything was terrible” (I don’t know about you but I tend to rate myself very negatively). However, you may find that if you looked at your progress over time, you may actually be improving quite a bit.

The best way I have found for measuring your progress is using a scoreboard that I discovered from Tina Huang. She prepares a spreadsheet with all the days she plans to study and she will mark whether she was able to get the things done that she said she was going to do. In this way, she is able to clearly visualise in a chart how she has been progressing over time. This gives immediate, clear feedback on your progress and is essential to staying motivated in the long-term.