"I do not know how she got that information," said Sen. Dianne Feinstein, chair of the Senate Intelligence Committee, referring to Broadwell's October speech. "We should find out."

Feinstein said she would investigate why the FBI investigation that uncovered the extramarital affair did not notify oversight committees about its investigation into Petraeus after the bureau determined he was having a secret and risky extramarital affair.

Fran Townsend, a former Bush Homeland security adviser and CNN national security contributor, agrees the situation lends itself for the potential that security would be breached.

"Is there the potential? Yes of course. I think there's no indication that there's a problem here. There's no indication this involved classified material from him to her."

She also said she thinks that Broadwell is "playing the journalists' game."

"If she's in an intimate relationship with the CIA director, he's not her only source." And, she might have heard the October remarks, made at the University of Denver, in passing from someone else.

As investigators try to determine who said what to whom, it might end up as a "he said, she said." And as a result, nothing will be firmed down or resolved.

"I worry about the distraction that this now becomes for Congress and the media," Townsend said.

Tom Fuentes, a former FBI assistant director and CNN contributor, said there are always potential worries about having appropriate relations and the potential of spies. In this situation, Petraeus is "very experienced" and knows about security clearances. Broadwell has scores of military contacts and is an author and academic.

But what if someone burglarized her house, for example, and found classified information there. Could she be blackmailed?

"That's the worry," he said.