The Justice Department has opened a formal investigation of the advertising deal struck last month between Google and Yahoo, The Washington Post reported on its Web site Tuesday evening, citing sources close to the inquiry.