How do I change my sidebar to show group/app icons and names?

Right-click on the sidebar to reveal a pop-up menu of display options; Regular, Compact, Tiny and List.
Clicking on 'list' will display the group names alongside your sidebar apps.
To change the name of your group, right-click on the icon and go to Settings > Group Settings, and look for Display Name.