Treasury yields climbed on Friday amid concerns about the possibility of higher inflation.

The yield on the benchmark 10-year Treasury note advanced to 1.34% at around 3:45 p.m. ET, while the yield on the 30-year Treasury bond rose to 2.14%. Yields move inversely to prices.

Treasury Secretary Janet Yellen told CNBC on Thursday that she still thinks President Joe Biden's $1.9 trillion stimulus package is needed to get the U.S. economy back to full strength.

"We think it's very important to have a big package [that] addresses the pain this has caused – 15 million Americans behind on their rent, 24 million adults and 12 million children who don't have enough to eat, small businesses failing," Yellen said.

She said her support for the package is despite signs that momentum is building in the first two months of 2021 and investors' fears inflation could return.

"Inflation has been very low for over a decade, and you know it's a risk, but it's a risk that the Federal Reserve and others have tools to address," she said. "The greater risk is of scarring the people, having this pandemic take a permanent lifelong toll on their lives and livelihoods."

The Treasury secretary also said that any future tax increases proposed to help offset the cost of Biden's big-ticket infrastructure and investment plans, expected later this year, will phase in gradually.

Yields tend to rise with inflation expectations as bond investors start to believe central banks will take their foot off the gas and reduce their asset purchases.

There are no auctions scheduled to be held Friday.

— CNBC's Tom Franck and Elliot Smith contributed to this report.