I think the German high court nailed down the problem very well:
Verifiability
The German constitutional court ruled about any kind of voting devices:
The usage of voting devices [...], is in compliance with the constitutional requirements only, if the essential steps of the voting and counting can be verified reliably and without expert knowledge.
The verdict was motivated by a case on traditional voting computers, but the ruling is more general: Those voting computers consist of a keyboard and a display. The voter presses the key for the party of his or her choice. The vote will be displayed and the voter has to confirm it.
But there is no way a voter can verify that the party he selected and saw on the screen, is the party his vote is counted for. The government tried to argue that the voting computers had been verified by an official institution and sealed to prevent manipulations.
But the court made it very clear, that "verified by experts" is not good enough, to fulfill the requirements of democratic elections.
Note: It makes no difference whether the voting is done using a foreign voting computer or a computer owned by the voter: People are not able to verify that their own computer is working correctly either. This already starts at understanding an open source voting software, but of course this goes down all the way to the microcode within the hardware.
Anonymity and non-provability
The ruling was about verifiability by non-experts only. Of course any solution to this issue, must not violate the other requirements of democratic elections.
For example verifying votes could be done by publishing a complete list of all voters with addresses and votes. But democratic elections requires anonymity to protect the voters and non-provability to prevent selling of votes.
There are some smart concepts of not publish the information for everyone to see, but still allowing the voter to verify his vote. But those concept are not easily understood by average people because of the complex math behind it. So we are back at square one.
Furthermore they cannot satisfy non-provability and provability at the same time. But if they don't allow someone to prove, that his vote was not counted the way it was supposed to be, it is quite likely that a number of people will call fraud after every election.