Treasury yields fell and prices increased on Tuesday after the U.S. halt of the Johnson & Johnson vaccine caused a bit of a flight to safety.

Also aiding the drop was a key inflation report that came in only slightly higher than expected and not as bad as some feared. 

The yield on the benchmark 10-year Treasury note dipped about 6 basis points to 1.618% at around 4:20 p.m. ET. The yield on the 30-year Treasury bond fell about 5 basis points to 2.301%. Yields move inversely to prices. One basis points equals 0.01%.

Consumer prices rose March, the Labor Department reported Tuesday. The consumer price index rose 0.6% from the previous month but 2.6% from the same period a year ago. The year over year gain is the highest since August 2018.

Economists polled by Dow Jones are projecting the headline index to rise by 0.5% month-over-month and 2.5% year-over-year.

Yields got hit when the Food and Drug Administration said it is asking states to pause administering J&J's Covid-19 vaccine after six people in the U.S. developed a rare disorder involving blood clots. The FDA said the recommendation is "out of an abundance of caution."

Treasury yields have jumped from just below 1% since the end of January, over fears of inflation rising as the U.S. economy recovers from the coronavirus pandemic.

Eric Lonergan, fund manager at M&G, told CNBC's "Squawk Box Europe" Tuesday that he believed bond markets had now "priced in a kind of post-crisis normalization" with the recent jump in yields.

He also said that an anticipated rise in inflation had been "so well-flagged now, (that) everybody is expecting some kind of temporary increase."

— CNBC's Patti Domm contributed to this report.