The question goes: "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?"

Is your knee-jerk response that the ball costs ten cents? That is a common answer, but also an incorrect one.

If the ball costs ten cents then the bat would cost $1.10, which would bring the total to $1.20. The correct answer is the ball costs 5 cents and the bat $1.05. 

This question is part of the Cognitive Reflection Test, CRT, which was first described in 2005 by psychologist Shane Fredrick. Fredrick wanted to examine how people fight, or don't fight, their intuitive thinking.

The original test contained two additional questions:

• If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
• In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?

Research about whether the test measures cognitive ability, or intelligence, has been mixed. 

However, why so many people get it wrong is due to a psychological trap we all fall into sometimes. This same psychological trap can hinder our ability to make sound decisions.

Why so many people get this math problem wrong

During the cognitive process, one type of activity is System 1 and the other is System 2. System 1 is your initial reaction, which is often made without reflection. System 2 is when you use conscious thought and effort. For system 2 to kick in, one must realize that System 1 did not net them the right answer. 

For some people the intuitive answer is the ball costs 10 cents. In order to come to the correct answer you need to realize that system 1 didn't work, and assess the problem again. 

Giving people another chance to solve a problem, though, doesn't automatically mean that they will get it right.

Even when given two chances, many people remain loyal to their knee-jerk answer, according to a recent study. In the experiment, researchers gave participants 50 versions of the bat-and-ball problem.

Participants had to first give their System 1 answer, or their initial hunch, and then were allowed to give a System 2 answer, which was supposed to be more thought-through.

"Results showed that both people's first hunches and the responses they gave after deliberation predominantly remained biased from start to finish," the study reads. "But in the rare cases in which participants did learn to correct themselves, they immediately managed to apply the solution strategy and gave a correct hunch on the subsequent problems." 

While making decisions, it's important to not always go with your hunch. Re-evaluate your choice, even if your intuition is telling you that you made the right one, and reflect on whether you are actually analyzing the problem or just looking for the easiest answer.

DON'T MISS: Want to be smarter and more successful with your money, work & life? Sign up for our new newsletter!

Get CNBC's free Warren Buffett Guide to Investing, which distills the billionaire's No. 1 best piece of advice for regular investors, do's and don'ts, and three key investing principles into a clear and simple guidebook.