For the past five years, investors have gotten drunk off the Federal Reserves easy-money punch bowl.

The Feds stimulus program, known as quantitative easing or QE, left investors with beer goggles on. Stocks and bonds that normally would look unattractive suddenly looked pretty good.

Now that Janet Yellen is getting ready to remove the punch bowl, investors are being forced to take a more critical look at their portfolios.

Everything looks good under the drug of QE, but once QE goes away those goggles start to clear up. Those warts and blemishes start to matter. Risk appetites change, explains Peter Boockvar, the Lindsey Group market analyst who came up with the beer goggles analogy back in January.

Even some members of the Fed admit the analogy works. Richard Fisher, the head of the Dallas Fed who has been critical of QE, used it in a speech earlier this year.

Investors have been dumping stocks found in the previously red hot Russell 2000, sending that basket of smaller-sized stocks into correction mode last week. Thats Wall Street code for a 10% decline from a previous high.

Smaller stocks are riskier than their larger cousins because their shares change hands less frequently, making it more difficult to buy and sell them. Theyre also more vulnerable to swings in the economy and credit markets than big stocks like Apple or Google.

The same sobering up process has hit so-called momentum stocks. High-flyers like Tesla, El Pollo Loco and Netflix are all hovering well below their 52-week highs. Even red hot GoPro started responding to the laws of gravity last week.

That doesnt necessarily mean these are lousy stocks; just that investors realize they may take a breather as less of the Feds easy money is sloshing around the financial system.

Even the credit markets are seeing the beer goggles impact. Junk bonds have taken a hit in recent weeks as Wall Street bets on which companies will be healthy enough to roll over their debt once interest rates begin to climb.

Reshaped risk and reward: The Feds emergency actions transformed the risk and reward of virtually all asset classes. Thats because with interest rates at zero, cash in the bank earns virtually nothing. It actually loses value due to inflation.

To avoid losing money, investors are forced to put more of their money in riskier assets like stocks and bonds. That helps explain one of the driving forces for the bull market in stocks and why the US government can borrow money for 30 years at just 3%.

But now the Fed is hitting the brakes on QE and is poised to lift interest rates off the floor next year.

So what should investors do in this changing environment?

Boockvar believes the bulls party days may soon be over.

People should brace for a correction. You could easily have a bear market if the Fed starts to lose control, he said.

Boockvar argues that the Feds easy-money policies only pulled forward a lot of asset price gains.

They want you to buy a car today instead of tomorrow. They want you to buy a stock today instead of tomorrow. When that drug goes away, you need to give back some of those gains that were artificially gotten, he said.

Others believe the party could still go on, just at more hushed tones and only for some parts of the stock market.

David Lebovitz, a global market strategist at JPMorgan Asset Management, expects large-cap stocks to fare better than small caps because they have better access to credit.

He especially likes stocks with exposure to the IT, energy and financial sectors.

The point is, blindly throwing money into the stock market might not be the best play once QE ends.

Were at a point in the cycle where differentiation is much more important. Its going to be much more about security selection, Lebovitz said.