As far as the possibility of backward compatibility goes:
PS1: Almost 100% certain it will be there, since PS2 and PS3 have it. This will be emulated in software, just like the PS3 has always done it.
PS2: About 95% chance of this happening. If we get PS2, it will be emulated in software. They're already selling PS2 classics that are modified to run on all PS3 machines (not just the early models. If the PS4 has good PS2 emulation, then they can quickly amass a PS2 classic library without having to go through a long modification process.
PS3: It doesn't look very likely that we'll get traditional backward compatibility, especially if they aren't including at least some portion of the PS3 chipset. One possibility is that they could use Gaikai for online streaming of PS3 games, as other people have mentioned. Another possibility, though, is that the system includes some sort of interface that allows an expansion module to be connected. The module would contain the parts of the PS3 chipset that cannot be easily emulated. By doing this, they can sell the basic PS4 system at a low price, then for an additional fee (for example, $100) allow people to add hardware-level PS3 compatibility to the system.
To sum it up: Both PS1 & PS2 would be emulated through software, while PS3 would be provided either through Gaikai or an optional expansion module.