The jobs picture improved slightly in July, but its impact on the housing recovery is more murky.

Mortgage bankers shed 1,200 jobs, as their refinance business has dropped dramatically due to higher rates. The unemployment rate for young adults rose to 7.8 percent, with just 74.8 percent of them working, according to the Bureau of Labor Statistics. That is the lowest share in a year.

(Read more: Jobs growth misses high hopes; rate drops to 7.3% )

“Without jobs, fewer young adults will buy, rent, or even move out of their parents’ homes, which holds back future household formation and longer-term demand for new construction,” noted Jed Kolko, chief economist for Trulia.

On the other hand, large downward revisions in overall jobs in July kept mortgage rates from rising even further. Conforming loan rates are tied to mortgage-backed-securities, or MBS, which tend to correlate with U.S. Treasuries.

Conforming loans are those backed by Fannie Mae, Freddie Mac or other government agencies. Their limit is $417,000 but can be as high as $625,500 in high-cost housing markets.

“Rates are most of the way back down to yesterday morning’s levels,” said Matthew Graham, chief operating officer of Mortgage News Daily. “Not all lenders are out with rate sheets yet, but those who are have moved back to 4.75 percent from 4.875 percent yesterday. It’s a bit of a consolation prize for now, as rates are still at their highest levels of the year with the exception of yesterday.”

Had the jobs report been stellar, Government bond yields —and, consequently, mortgage rates— could have moved even higher as investors may have been more convinced the Federal Reserve would begin tapering asset purchases at their September meeting.

The negative revisions to the jobs numbers leaves enough doubt about that for interest rates to catch their breath just a bit.

(Read more: Jobs data good enough for tapering, just ‘taper light’)

“The jobs report will lead rates lower through the weekend. When taper talk resumes, however, rates will rise. This could happen as soon as Monday. San Francisco Fed President John Williams has a public speech planned. Hawkish remarks will move rates higher,” said Dan Green, loan officer with Waterstone Mortgage and author of The Mortgage Reports.

Green also pointed out that U.S. mortgage rates are changing at the fastest pace in more than 4 years, “challenging home buyers and refinancing households in search of ‘today’s lowest mortgage rate’.”

(Read more: Home prices push past rising rates)

In another baffling twist to home buyers, conforming loan rates are now higher than those for pricier, so-called “jumbo” loans. Historically, they are lower by about 25 basis points. This has been the case for the past two weeks, as Fannie Mae and Freddie Mac layer on extra fees, a strategy linked to their mandate of shrinking their portfolios.

At the same time, low borrowing costs for the big banks have kept them rolling in cash and more willing to lend out of their own portfolios into the jumbo market. As rates rise in general, so too can their profits. These loans, however, are only for buyers with large down payments and pristine credit.

(Read more: Map: Tracking the recovery)

Rate volatility will undoubtedly persist throughout this fall, and different loan products will undergo odd permutations in this new finance frontier. That said, rates will rise.

“The housing industry’s hope is that these less-than-stellar job reports are going to delay the [Fed] tapering, the next 25 or 50 basis point rise in rates, but we are only talking about months here,” says Jaret Seiberg of Guggenheim Partners. “We firmly see rates going higher.”

—By CNBC’s Diana Olick. Follow her on Twitter @Diana_Olick.