An Act Boss is simply the boss monster you must defeat at the end of each Act.
Diablo II
In Diablo II they appear in the following order:
Their deaths grant you access to the following Act. Depending whether you have Diablo II: Lord of Destruction or not, you have to kill Normal Baal or Diablo to have access to Nightmare, and you have to kill Nightmare Baal or Diablo to have access to Hell.
Diablo III
In Diablo III they appear in the following order:
Like in Diablo II, their deaths grant you access to the following Act. Once a character kills Diablo, Master difficulty will unlock for all characters on that account.
Guaranteed Legendaries
The first time a character level 1-59 kills an act boss, it is guaranteed to drop at least one legendary item. Once the game is completed with that character, they can choose to "Reset Quests". Doing so will again guarantee one legendary item for the first kill of each act boss. Once a character reaches level 60 or higher, only Malthael will drop a guaranteed legendary item. If the player does not have Reaper of Souls, this guaranteed drop moves to Diablo.