I assume you have enabled SSL so the site is accessible via HTTPS.

Assuming you haven't changed the configuration, go ahead and type

unlink /etc/apache2/sites-enabled/000-default

This would remove the symlink named 000-default, which points to the default vhost found in sites-available. If you have altered the config, make sure to remove any vhost listening on *:80.

This first step is kind of straight-forward. You want Apache2 to stop listening on port 80, you remove any vhost on that port. But that's not all.

Open up ports.conf

nano /etc/apache2/ports.conf

Comment out (put # at the beginning of the line)

NameVirtualHost *:80
Listen 80

Go ahead and restart the web server and load the changes

service apache2 restart

Now Apache2 should listen on port 443, and 443 only.