Vagrant won't let me access Virtual Box's Guest window

Question

I am a big Linux and Vagrant newbie, so please excuse my inexperience.

So, I have a Slackware vagrant box. When I edit the Vagrantfile with this code:

config.vm.provider "virtualbox" do |v|
  v.gui = true
end

Typing in vagrant up after initializing said box will fire up a VB guest window, where I can use it like any other VM. No problem.

However, is there a way to be able to SSH into it with the Vagrant SSH command and fire up the VM window from there without having to edit my Vagrant File? I'll ssh into my VagrantBox, and my Virtual Box client will show in the preview window what would normally appear on the guest window, but I cannot directly access it. I can type in startxfce4, and the preview window will show XFCE firing up, but a Virtual Box guest window won't show up, so I can't do anything with the GUI.

Edit: Does this have something to do with Guest Additions? I'm on a older version, 4.1, and the box has version 4.2.16 but I'm using ChrUbuntu on an Acer C7, and this has been the most stable VB has ran on my machine. (My primary machines are down at the moment, so I can't test this problem out there.)