I accedently installed the desktop packages for Debian in a virtual machine, when all I wanted was a clean testing box. I've had some troubles with the network managers, but mainly - why boot into GNOME when you have no need for a point and click interface? Such a waste.
A quick, and perhaps dirty solution would be to run the following:
sudo apt-get autoremove gdm3
sudo apt-get autoremove –purge gnome*
Works fine in a virtual environment. This is a quick fix I wouldn't use on an important server. Don't blame me if something breaks :3.