Even simpler: when caloric intake is below energy output, the result is weight loss. When caloric intake exceeds energy output, the result is weight gain.
This is in theory correct, but there's many factors which decides how much your body will uptake, store and burn. It is possible to eat less but gain weight. It is also possible to gain weight and loose fat. I don't know what this obsession with weight comes from as it's generally a very poor measurement of fitness.
However, by far the most common explanation for "I started eating less, counting calories, and I gained weight" is not a tremendous increase in metabolic efficiency, but a combination of:
1) spontaneous fidgeting and unscheduled exercise dropping off (sitting in your chair all day without getting your normal couple trips to the water cooler/bathroom, not shopping today because you're tired) - this is called "NEAT" if you want to Google
2) self-delusion as to the degree of reduced food intake
The latter is the most likely cause of severely failing diets.
No. Unless you cut energy expenditure as well, the energy has
to come from somewhere. If you are consuming 1500 Kcal and using 2000 Kcal, you will loose weight. Thermodynamics is not out to get you.
It might be possible to eat less but gain fat, however.
Your body can regulate how efficiently it stores what you are putting into it. If you restrict your diet too much, your body will think that food is scarce, and it will store more.
For example, several years ago I wanted to lose weight so I was trying to eat as little as possible and skipping breakfast. My weight stayed mostly constant. Then I decided to start eating breakfast and lost about 10 pounds very quickly. (I know you might think that by skipping breakfast I was eating a bigger lunch, or something like that, but it wasn't the case. I was in college on a very regular schedule, eating the same thing for lunch from the same food truck every day).
I realize this is just one person's anecdotal evidence, but from what I understand this is very common among people who skip breakfast in trying to restrict their caloric intake.
This is why experts recommend eating many small but healthy meals throughout the day, and never skipping out on breakfast.
You can and do change your energy expenditure, sometimes dramatically. I've lost 1-2 kg over the course of a weekend of being sick before, but that's just an extreme. All of "eat less loose weight" strategies work only if you consider amount eaten and energy expended to be independent. Which, to me, sounds incredibly unlikely considering how fine-tuned we are in every other respect.
When all said and done, my personal weight loss method is still eating less: but eating dramatically less then normal for relatively short periods (4-10 weeks). I'm a bit scared to think about everything my body is shutting down in this time. For starters, it takes about a week for my brain to catch up with the changed diet.
The human body is not 100% efficient. The efficiency of a human body can change. The amount of Kcal in a food item measured in a lab is not necessarily going to be the same amount a human absorbs.