To delete a task, first click its number to select it.

Then, click the Edit menu and click Delete Task, or press the Delete key on your keyboard.

Voila! The task is gone.