You'll need to be sure that you have the libvirt guest utilities installed to your guest operating system, to begin.
Also, in order for copy/paste to work, you'll need a SPICE guest agent socket device in addition to a VirtIO Serial device assigned to the VM, which will allow virt-viewer and most other SPICE clients to pipe copy/paste over a virtual serial connection, which the guest agent will hande. Defining these devices may have already been done for you, especially if you use virt-manager to make these VMs (virt-manager assumes a GUI install of most guests).
In order to install the guest agent on a Debian based GNU/Linux guest machine, you'll need to run:
sudo apt install spice-vdagent
In order to install the guest agent on a Windows machine, you must install the package which can be found here: https://www.spice-space.org/download/binaries/spice-guest-tools/
And finally, you will need to install VirtIO drivers in Windows in order to get most advanced functionality. This can be accomplished by adding drivers from the Win-virtio ISO (which you'll have to attach to the guest). A link to those: https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers
As you've pointed out in the comment below, you must ensure that the spice agent is running in the guest before this will function fully. A reboot is a sure way to get that agent running.