1. In the Task bar, click the Show hidden icon  button to display the icons box.

2. Find and right click the Outlook icon, uncheck Hide When Minimized option from the list.

From now on, when you minimizing the Outlook, the icon still be highlighted in the task bar and cause no confusion of closing for you anymore.