How to search for all labeled files, except those in some folders?

Question

I want to add a saved search to my sidebar that contains all labeled files on my Mac, except files that are in certain folders. Files that are in subfolders of the excluded folders shouldn’t be excluded from the search though.

Here’s an example to make it a little bit clearer:

• Files and folders that are directly in ~/Downloads/ should be excluded from the search.
• Files that are in subfolders of ~/Downloads/ and are labeled, like ~/Downloads/foo/ or ~/Downloads/bar/ shouldn’t be excluded.
• The list of subfolders is dynamic.

I don’t want to use a third-party app for this task.