You can't compare the S&P 500 with a fund that, at age 50, would hold 25% in bonds. Of course the TDF will lose.
The function of gradually transitioning to bonds is to hedge against inflation, lock in profits, and buffer against market volatility. Those are useless for someone just starting out in their 20s — which is why TDFs start out with very little bonds — but important to someone whose time horizon is just 15 years. Someone at age 50 should be extremely careful about holding just the S&P.
Also, keep in mind that the equity portion of Vanguard's TDFs are 60/40 US/international, because that's what Vanguard thinks gives you best diversification [1]. Depending on the time period you measure, international does either better or worse than the US [2].
You may be interested in reading about Vanguard's target date fund philosophy. [3]
I wasn't stating my question clearly, apologies - I'm on board with the reasons and the theories (and do not argue), but here's where my brain is having a problem - why not invest in the index fund until you're 65 and then roll it over into a TDF at 65? You get more bang for the buck even paying the capital gains tax after 15 years and taking the money you earned and have more TDF buying power. You (me) do not need the benefits of the TDF until I actually retire (well I won't, am I common?).
Parameters for my math: already-income-taxed dollars; $1200 one-time investment in 2006 (no re-investments of dividends, etc.) and capital gains of 15% (middle tier). Just to make the math round nicely and account for gains tax for a rollover and forum comment. :)
So if our sample 45yo person had placed their $1200 into VFINX in 2006, they could have sold it in 2020, paid 15% in gains tax and purchased ...eh, let's say 78 shares of VTWNX, a +16 share gain over just buy-and-hold of 60x VTWNX until 2020. In 2020 this sample person is now 60 and eligible to withdraw without penalty (thinking a Roth IRA here, my model). Remember that this target date is marketed at and intended for people who retire on or close to that year, so our sample person who buys it in 2006 is 45yo (the target audience). It is not expected this fund would have been purchased by a 20yo in 2006, logically.
This is where I'm not following why it was better for this person to buy and hold VTWNX for 15y instead of increasing gains with VFINX first, then rolling it over into that more "bond-like" scenario later. Feels like I'm leaving money on the table as 40 years of market data shows the index @8.25% just keeps going up over time (even when you lose like in 2009 with a low of $68, the loss is still higher than 1995 value of $55 without inflation adjustments).
If you think the S&P will hold up until the day you retire, sure.
In that case, I wouldn't use a TDF (because you have no target date); I'd move to a balanced fund such as Vanguard LifeStrategy.
But a pure equity portfolio is considered very aggressive and risky for someone close to retirement. What if another 2008 happens?
Bonds reduce volatility. Look at the mid-March 2020 drop. At the lowest, the S&P was -32% YTD. BND's lowest point was -4% and was positive 2 weeks later. S&P didn't pass zero until August, more than 5 months later.
Bonds also allow you to lock in your gains. Being less volatile, the bond portion is much safer than stocks. Your main enemy there is inflation, which is why many TDFs supplement with US TIPS.
I understand if you want to be aggressive. That's fine, and you don't have to use a TDF. I know J. L. Collins (who's retired) caps his bond allocation to 20%.
The function of gradually transitioning to bonds is to hedge against inflation, lock in profits, and buffer against market volatility. Those are useless for someone just starting out in their 20s — which is why TDFs start out with very little bonds — but important to someone whose time horizon is just 15 years. Someone at age 50 should be extremely careful about holding just the S&P.
Also, keep in mind that the equity portion of Vanguard's TDFs are 60/40 US/international, because that's what Vanguard thinks gives you best diversification [1]. Depending on the time period you measure, international does either better or worse than the US [2].
You may be interested in reading about Vanguard's target date fund philosophy. [3]
[1] https://www.vanguard.com/pdf/ISGGEB.pdf
[2] https://www.fidelity.com/viewpoints/investing-ideas/internat...
[3] https://personal.vanguard.com/pdf/ISGTDF.pdf